面向基础系列之---Java网络编程---网络链接组件的使用（URLConnection）

URL主要功能用于读取服务端或者目标地址的数据，可是要具体对一个请求的元数据进行解析，就无能为力了。这个时候，URLConnection就是一个很好的切入口。这东西不只仅能读取数据，还能对元数据进行读取，还能读取header（头header仍是很重要的，互联网开发中，每每咱们就是要header里面的数据），而且，URLConnection还能使用各类的HTTP方法（POST/GET/OPTIONS/PUT/DELETE）往服务端发数据。本章我不介绍太多，尽可能精简。

1、构建与读

总体上使用URLConnection的基本步骤以下：

1. 构造一个URL对象
2. 调用这个URL的openConnection()获取一个对应的URLConnection对象
3. 配置这个URLConnection
4. 读取首部字段
5. 获取输入流并读取数据
6. 得到输出流并写入数据
7. 关闭链接

基本的代码片断以下：

try{
    URL u = new URL("http://www.baidu.com");
    URLConnection conn = u.openConnection();
    // 从URL读取
} catch(MalformedURLException ex){
    System.err.println(ex);
} catch(IOException ex){
    System.err.println(ex)
}

一、内部一些简单原理

• URLConnection是一个抽象类，只有一个方法没有实现：public void connect() throws IOException
• 一些常见的实现类：
  • sun.net.www.protocol.file.FileURLConnection：文件名相关
  • sun.net.www.protocol.http.HttpURLConnection：网络相关
• 建立URLConnection以后，不进行connect的调用，在第一次要进行数据通讯的时候，才调用，例如：getInputStream、getContent、getHeaderField等

二、读取服务器的数据

public class NetworkMain {

    public static void main(String[] args) {
        try {
            URL url = new URL("https://www.baidu.com");
            URLConnection urlConnection = url.openConnection();

            try (InputStream inputStream = urlConnection.getInputStream();){
                InputStream buffer = new BufferedInputStream(inputStream);
                InputStreamReader reader = new InputStreamReader(buffer);
                int c;
                while ((c=reader.read()) != -1){
                    System.out.print((char) c);
                }
            }
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
}

URL与URLConnection的区别：

• URLConnection提供了对HTTP首部的读取
• URLConnection能够配置发送给服务器的请求
• URLConnection出了读取，还能写入，箱服务器

三、首部与读取

下面是一个百度首页获取的header具体信息：

Accept-Ranges:[bytes]
null:[HTTP/1.1 200 OK]
Server:[bfe/1.0.8.18]
Etag:["58860402-98b"]
Cache-Control:[private, no-cache, no-store, proxy-revalidate, no-transform]
Connection:[Keep-Alive]
Set-Cookie:[BDORZ=27315; max-age=86400; domain=.baidu.com; path=/]
Pragma:[no-cache]
Last-Modified:[Mon, 23 Jan 2017 13:24:18 GMT]
Content-Length:[2443]
Date:[Thu, 13 Sep 2018 09:51:05 GMT]
Content-Type:[text/html]

获取的代码以下：

public class NetworkMain {

    public static void main(String[] args) {
        try {
            URL url = new URL("https://www.baidu.com");
            URLConnection urlConnection = url.openConnection();
            Map<String, List<String>> headerFields = urlConnection.getHeaderFields();
            for(Map.Entry<String,List<String>> entry : headerFields.entrySet()){
                System.out.println(entry.getKey()+":"+entry.getValue().toString());
            }
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
}

a、Content-type

返回响应主题的MIME（(Multipurpose Internet Mail Extensions)多用途互联网邮件扩展类型）。其实就是内容的类型和编码方式

• 没指定不会抛异常，直接返回null
• text/html不指定编码方式，默认使用ISO-8859-1，是http默认编码方式
• 其余经常使用的类型还有：text/plain、image/gif、application/xml、image/jpeg
• 可经过getContentEncoding方法进行获取编码方式，不指定会返回null

b、Content-length

获取内容的总共的字节大小，若是没有content-length头，getContentLength()方法就返回-1

• Java7中增长了个getContentLengthLong方法，返回long，防止超出int最大值
• http下载一个二进制文件，最好使用getContentLength方法来判断何时结束InputStream对象

c、Date

指出文件什么时候发送的

d、Expires

指示什么时候从缓存中删除文档，若是没有这个header，getExpireation方法放回0，表示永远不会过时

e、Last-Modified

文档最后修改时间，没有这个header的话，getLastModified方法返回0

2、缓存

缓存是永恒的话题~好吧，web浏览器的缓存，也是一个能屠龙的功能。这小节会介绍下若是使用web缓存，与Java中设置缓存的几个类

一、如何设置header使之可以缓存

通常来讲GET的HTTP请求都会缓存，也应该缓存，可是POST请求就不该该缓存。固然这些均可以经过header进行调整：

• Expires首部（HTTP1.0）指示能够缓存这个资源，知道指定的时间为止
• Cache-control首部（HTTP1.1）细粒度的缓存控制，若是这个和expires首部都有，会以这个首部为主，多个cache-control是被容许的：
  • Max-age=[second]：从如今到缓存项过时以前的秒数
  • s-maxage=[seconds]：从如今起，知道缓存项再共享缓存中过时以前的秒数。私有缓存能够将缓存项保存更长时间
  • Public：能够缓存一个通过认证的响应。不然已认证的响应不能缓存
  • Private：仅单个用户缓存能够保存响应，而共享缓存不该该保存
  • No-cache：缓存项仍然能够缓存，不过客户端在每次访问时都要用一个Etag或者Last-modified头从新验证响应
  • no-store：无论怎样都不缓存
• Last-modified：最后一次修改日期。客户端可使用一个HEAD请求来检查这个日期，只有当本地缓存的日期早于这个值，才会真正执行GET请求
• Etag：资源的惟一标识。HEAD请求访问这个Etag服务端的值，只有与本地的Etag值不一样的状况下，说明缓存失效了，才会调用GET请求

二、Java的Web缓存

默认请款下，直接使用URL请求资源的时候，Java是不进行缓存的，要默认实现几个类来增长Java对Web请求的缓存功能：

ResponseCache//设置默认缓存策略的对象
CacheRequest//设置请求的对象
CacheResponse//设置回复请求的对象

一个简简单单的实现代码，稍微有点长，不过不难，其中还有对header中cache-control字段的解析对象构建，是一个不错的起步例子：

import java.io.*;
import java.net.*;
import java.util.*;
import java.util.concurrent.ConcurrentHashMap;

public class NetworkMain {

    public static class CacheControl {

        private Date maxAge = null;
        private Date sMaxAge = null;
        private boolean mustRevalidate = false;
        private boolean noCache = false;
        private boolean noStore = false;
        private boolean proxyRevalidate = false;
        private boolean publicCache = false;
        private boolean privateCache = false;

        public CacheControl(String s) {
            if (s == null || !s.contains(":")) {
                return; // default policy
            }

            String value = s.split(":")[1].trim();
            String[] components = value.split(",");

            Date now = new Date();
            for (String component : components) {
                try {
                    component = component.trim().toLowerCase(Locale.US);
                    if (component.startsWith("max-age=")) {
                        int secondsInTheFuture = Integer.parseInt(component.substring(8));
                        maxAge = new Date(now.getTime() + 1000 * secondsInTheFuture);
                    } else if (component.startsWith("s-maxage=")) {
                        int secondsInTheFuture = Integer.parseInt(component.substring(8));
                        sMaxAge = new Date(now.getTime() + 1000 * secondsInTheFuture);
                    } else if (component.equals("must-revalidate")) {
                        mustRevalidate = true;
                    } else if (component.equals("proxy-revalidate")) {
                        proxyRevalidate = true;
                    } else if (component.equals("no-cache")) {
                        noCache = true;
                    } else if (component.equals("public")) {
                        publicCache = true;
                    } else if (component.equals("private")) {
                        privateCache = true;
                    }
                } catch (RuntimeException ex) {
                    continue;
                }
            }
        }

        public Date getMaxAge() {
            return maxAge;
        }

        public Date getSharedMaxAge() {
            return sMaxAge;
        }

        public boolean mustRevalidate() {
            return mustRevalidate;
        }

        public boolean proxyRevalidate() {
            return proxyRevalidate;
        }

        public boolean noStore() {
            return noStore;
        }

        public boolean noCache() {
            return noCache;
        }

        public boolean publicCache() {
            return publicCache;
        }

        public boolean privateCache() {
            return privateCache;
        }
    }

    public static class SimpleCacheRequest extends CacheRequest {
        private ByteArrayOutputStream out = new ByteArrayOutputStream();


        @Override
        public OutputStream getBody() throws IOException {

            return out;
        }

        @Override
        public void abort() {
            out.reset();
        }

        public byte[] getData() {
            if (out.size() == 0) {
                return null;
            } else {
                return out.toByteArray();
            }
        }
    }

    public static class SimleCacheResponse extends CacheResponse {
        private final Map<String, List<String>> headers;
        private final SimpleCacheRequest request;
        private final Date expires;
        private final CacheControl control;

        public SimleCacheResponse(SimpleCacheRequest request, URLConnection uc, CacheControl control) throws IOException {
            this.request = request;
            this.control = control;
            this.expires = new Date(uc.getExpiration());
            this.headers = Collections.unmodifiableMap(uc.getHeaderFields());
        }

        @Override
        public InputStream getBody() {
            return new ByteArrayInputStream(request.getData());
        }

        @Override
        public Map<String, List<String>> getHeaders()
                throws IOException {
            return headers;
        }

        public CacheControl getControl() {
            return control;
        }

        public boolean isExpired() {
            Date now = new Date();
            if (control.getMaxAge().before(now)) return true;
            else if (expires != null && control.getMaxAge() != null) {
                return expires.before(now);
            } else {
                return false;
            }
        }
    }
    public static class MemoryCache extends ResponseCache {

        private final Map<URI, SimleCacheResponse> responses
                = new ConcurrentHashMap<URI, SimleCacheResponse>();
        private final int maxEntries;

        public MemoryCache() {
            this(100);
        }

        public MemoryCache(int maxEntries) {
            this.maxEntries = maxEntries;
        }

        @Override
        public CacheRequest put(URI uri, URLConnection conn)
                throws IOException {

            if (responses.size() >= maxEntries) return null;

            CacheControl control = new CacheControl(conn.getHeaderField("Cache-Control"));
            if (control.noStore()) {
                return null;
            } else if (!conn.getHeaderField(0).startsWith("GET ")) {
                // only cache GET
                return null;
            }

            SimpleCacheRequest request = new SimpleCacheRequest();
            SimleCacheResponse response = new SimleCacheResponse(request, conn, control);

            responses.put(uri, response);
            return request;
        }

        @Override
        public CacheResponse get(URI uri, String requestMethod,
                                 Map<String, List<String>> requestHeaders)
                throws IOException {

            if ("GET".equals(requestMethod)) {
                SimleCacheResponse response = responses.get(uri);
                // check expiration date
                if (response != null && response.isExpired()) {
                    responses.remove(response);
                    response = null;
                }
                return response;
            } else {
                return null;
            }
        }
    }

    public static void main(String[] args) {
        ResponseCache.setDefault(new MemoryCache());
        try {
            URL url = new URL("https://www.baidu.com");
            URLConnection urlConnection = url.openConnection();
            Map<String, List<String>> headerFields = urlConnection.getHeaderFields();
            for (Map.Entry<String, List<String>> entry : headerFields.entrySet()) {
                System.out.println(entry.getKey() + ":" + entry.getValue().toString());
            }
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
}

3、一些链接配置项

URLConnection类有7个保护的字段，定义了客户端如何向服务端做出请求，JDK源码中，对这些配置项作了很好的说明，直接读英文无压力，我就很少说了：

/**
     * The URL represents the remote object on the World Wide Web to
     * which this connection is opened.
     * <p>
     * The value of this field can be accessed by the
     * {@code getURL} method.
     * <p>
     * The default value of this variable is the value of the URL
     * argument in the {@code URLConnection} constructor.
     *
     * @see     java.net.URLConnection#getURL()
     * @see     java.net.URLConnection#url
     */
	protected URL url;

   	/**
     * This variable is set by the {@code setDoInput} method. Its
     * value is returned by the {@code getDoInput} method.
     * <p>
     * A URL connection can be used for input and/or output. Setting the
     * {@code doInput} flag to {@code true} indicates that
     * the application intends to read data from the URL connection.
     * <p>
     * The default value of this field is {@code true}.
     *
     * @see     java.net.URLConnection#getDoInput()
     * @see     java.net.URLConnection#setDoInput(boolean)
     */
    protected boolean doInput = true;

   /**
     * This variable is set by the {@code setDoOutput} method. Its
     * value is returned by the {@code getDoOutput} method.
     * <p>
     * A URL connection can be used for input and/or output. Setting the
     * {@code doOutput} flag to {@code true} indicates
     * that the application intends to write data to the URL connection.
     * <p>
     * The default value of this field is {@code false}.
     *
     * @see     java.net.URLConnection#getDoOutput()
     * @see     java.net.URLConnection#setDoOutput(boolean)
     */
    protected boolean doOutput = false;
   	/**
     * If {@code true}, this {@code URL} is being examined in
     * a context in which it makes sense to allow user interactions such
     * as popping up an authentication dialog. If {@code false},
     * then no user interaction is allowed.
     * <p>
     * The value of this field can be set by the
     * {@code setAllowUserInteraction} method.
     * Its value is returned by the
     * {@code getAllowUserInteraction} method.
     * Its default value is the value of the argument in the last invocation
     * of the {@code setDefaultAllowUserInteraction} method.
     *
     * @see     java.net.URLConnection#getAllowUserInteraction()
     * @see     java.net.URLConnection#setAllowUserInteraction(boolean)
     * @see     java.net.URLConnection#setDefaultAllowUserInteraction(boolean)
     */
    protected boolean allowUserInteraction = defaultAllowUserInteraction;
	/**
     * If {@code true}, the protocol is allowed to use caching
     * whenever it can. If {@code false}, the protocol must always
     * try to get a fresh copy of the object.
     * <p>
     * This field is set by the {@code setUseCaches} method. Its
     * value is returned by the {@code getUseCaches} method.
     * <p>
     * Its default value is the value given in the last invocation of the
     * {@code setDefaultUseCaches} method.
     *
     * @see     java.net.URLConnection#setUseCaches(boolean)
     * @see     java.net.URLConnection#getUseCaches()
     * @see     java.net.URLConnection#setDefaultUseCaches(boolean)
     */
	protected boolean useCaches = defaultUseCaches;

   	/**
     * Some protocols support skipping the fetching of the object unless
     * the object has been modified more recently than a certain time.
     * <p>
     * A nonzero value gives a time as the number of milliseconds since
     * January 1, 1970, GMT. The object is fetched only if it has been
     * modified more recently than that time.
     * <p>
     * This variable is set by the {@code setIfModifiedSince}
     * method. Its value is returned by the
     * {@code getIfModifiedSince} method.
     * <p>
     * The default value of this field is {@code 0}, indicating
     * that the fetching must always occur.
     *
     * @see     java.net.URLConnection#getIfModifiedSince()
     * @see     java.net.URLConnection#setIfModifiedSince(long)
     */
    protected long ifModifiedSince = 0;

   	/**
     * If {@code false}, this connection object has not created a
     * communications link to the specified URL. If {@code true},
     * the communications link has been established.
     */
    protected boolean connected = false;

对象中有相对应的set和get方法，通常若是在openConnection方法调用以后进行set，都会抛出IllegalStateException异常

4、向服务端写数据

这部分两块，写header，写内容

一、设置请求数据的header

这里设置header和前面的不同，前面是对服务端请求过来的数据进行header读取，这里会回写服务端的时候，对这个请求Request进行header添加的操做，主要用下面这几个方法：

public void setRequestProperty(String key, String value);//设置一个key对应的值，value能够逗号分隔设置多个
public void addRequestProperty(String key, String value);//对一个key的值进行添加值的操做

比较好玩的是，发现setRequestProperty的源码不难，能够看看，增长源码的亲密度

public abstract class URLConnection {
    ...
	public void setRequestProperty(String key, String value) {
        if (connected)
            throw new IllegalStateException("Already connected");
        if (key == null)
            throw new NullPointerException ("key is null");

        if (requests == null)
            requests = new MessageHeader();

        requests.set(key, value);
    }
    
    ...
}
public class MessageHeader {
    private String[] keys;
    private String[] values;
    private int nkeys;

	public synchronized void set(String var1, String var2) {
        int var3 = this.nkeys;

        do {
            --var3;
            if (var3 < 0) {
                this.add(var1, var2);
                return;
            }
        } while(!var1.equalsIgnoreCase(this.keys[var3]));

        this.values[var3] = var2;
    }
    public synchronized void add(String var1, String var2) {
        this.grow();
        this.keys[this.nkeys] = var1;
        this.values[this.nkeys] = var2;
        ++this.nkeys;
    }
    private void grow() {
        if (this.keys == null || this.nkeys >= this.keys.length) {
            String[] var1 = new String[this.nkeys + 4];
            String[] var2 = new String[this.nkeys + 4];
            if (this.keys != null) {
                //会发现底层JDK会使用这种，由于快速！
                System.arraycopy(this.keys, 0, var1, 0, this.nkeys);
            }

            if (this.values != null) {
                System.arraycopy(this.values, 0, var2, 0, this.nkeys);
            }

            this.keys = var1;
            this.values = var2;
        }

    }
}

二、POST写数据

其实对因而GET仍是POST写数据，Java的URLConnection会有个相似于自动判断的功能：

• 默认是GET
• 若是将doOutput参数置为true，使用OutputStream写数据，就是POST，会自动设置header
• 固然，有其余方法主动设置请求方法

下面是一个提交POST请求的小小例子：

public static void main(String[] args) {
    try {
        URL url = new URL("https://www.baidu.com");
        URLConnection urlConnection = url.openConnection();
        urlConnection.setDoOutput(true);
        OutputStream outputStream = urlConnection.getOutputStream();
        BufferedWriter bw = new BufferedWriter(new OutputStreamWriter(outputStream, "8859_1"));
        bw.write("lalalalallala");
        bw.flush();
        bw.close();
    } catch (IOException e) {
        e.printStackTrace();
    }
}

5、HttpURLConnection

默认如何URL请求是一个http的协议的话，返回的就是这个HttpURLConnection这个对象，他是URLConnection的抽象子类。使用public void setRequestMethod(String method) throws ProtocolException 方法来设置具体使用什么HTTP请求方法。下面几个经常使用的方法罗列：

• GET
• POST
• HEAD：常常用于获取最后修改时间以淘汰缓存
• PUT
• DELETE
• OPTIONS：跨域使用（重点），询问服务器支持哪些HTTP的方法
• TRACE：查看服务器和客户端之间的代理服务器作了哪些修改，能够ng配置查询使用