Volley源码学习笔记一

Android volley

---

Volley框架的官方架构图

从图中大概可以看出，框架中包含三种线程，主线程，内存线程，网路线程。网络线程可以是多个，主线程负责将请求按优先权顺序添加进任务队列，内存线程查看缓存中是否有此请求，有的话从缓存中获取返回数据回馈给主线程，否则将请求交给网络线程，网络线程多个任务线程（NetworkDispatcher）组成，这些任务线程同时启动，不停的从任务队列中获取待执行的任务，执行完毕后把返回结果回馈给主线程。

Volley框架HTTP请求流程图

从图中可以简单了解到，各类请求添加进请求队列，然后分发给网络线程（RequestDispatcher），通过HurlStack或者HttpClientStack执行网络请求，得到NetworkResponse返回，如果是错误，可以retry一次，否则直接返回给主线程错误信息，如果返回数据正确，就会解析成Response<T> 通过ResponseDelivery返回给主线程。

代码分析

首先是查看创建请求队列RequestQueue：

 /**
     * Creates a default instance of the worker pool and calls {@link RequestQueue#start()} on it.
     *
     * @param context A {@link Context} to use for creating the cache dir.
     * @param stack An {@link HttpStack} to use for the network, or null for default.
     * @return A started {@link RequestQueue} instance.
     */
    public static RequestQueue newRequestQueue(Context context, HttpStack stack) {
        //这里创建缓存目录
        File cacheDir = new File(context.getCacheDir(), DEFAULT_CACHE_DIR);
        //这里创建默认的useragent
        String userAgent = "volley/0";
        try {
            String packageName = context.getPackageName();
            PackageInfo info = context.getPackageManager().getPackageInfo(packageName, 0);
            userAgent = packageName + "/" + info.versionCode;
        } catch (NameNotFoundException e) {
        }
        //可以传入一个自定义的HttpStack,比如OkHttpClient
        if (stack == null) {
            if (Build.VERSION.SDK_INT >= 9) {
                stack = new HurlStack();
            } else {
                // Prior to Gingerbread, HttpUrlConnection was unreliable.
                // See: http://android-developers.blogspot.com/2011/09/androids-http-clients.html
                stack = new HttpClientStack(AndroidHttpClient.newInstance(userAgent));
            }
        }
        Network network = new BasicNetwork(stack);
        RequestQueue queue = new RequestQueue(new DiskBasedCache(cacheDir), network);
        queue.start();
        return queue;
    }

先看下HttpStack，这里的一个参数：

/**
 * An HTTP stack abstraction.
 */
public interface HttpStack {
    /**
     * Performs an HTTP request with the given parameters.
     *
     * <p>A GET request is sent if request.getPostBody() == null. A POST request is sent otherwise,
     * and the Content-Type header is set to request.getPostBodyContentType().</p>
     *
     * @param request the request to perform
     * @param additionalHeaders additional headers to be sent together with
     *         {@link Request#getHeaders()}
     * @return the HTTP response
     */
    public HttpResponse performRequest(Request<?> request, Map<String, String> additionalHeaders)
        throws IOException, AuthFailureError;
}

根据描述可以看出这是一个抽象的Http请求客户端，为了兼容不同的网络请求。定义了一个网络执行方法，返回的是HttpResponse类型。即不管用何种方式执行网络请求，返回格式需要封装成HttpResponse。

代码中网络请求的实现由两种类型，一种是Java原生的HttpURLConnection实现（HurlStack），一种是Apache的HttpClient实现（HttpClientStack），Volley会在android2.3以前使用HttpClient实现，在android2.3及以后使用HttpURLConnection实现，至于原因，官方的解释是：在Eclair和Froyo上Apache HTTP client拥有更少的bug，更好的稳定性，在Gingerbread以及以后的版本中，HttpURLConnection是最好的选择，它简单的api以及轻量级非常适合Android。压缩和缓存机制降低了网路使用，提高了速度、节省了电量。

这里可以简单的看下HurlStack和HttpClientStack对于网络执行方法performRequest的重写代码：

HurlStack：

 @Override
    public HttpResponse performRequest(Request<?> request, Map<String, String> additionalHeaders)
            throws IOException, AuthFailureError {
        String url = request.getUrl();
        HashMap<String, String> map = new HashMap<String, String>();
        map.putAll(request.getHeaders());
        map.putAll(additionalHeaders);
        if (mUrlRewriter != null) {
            String rewritten = mUrlRewriter.rewriteUrl(url);
            if (rewritten == null) {
                throw new IOException("URL blocked by rewriter: " + url);
            }
            url = rewritten;
        }
        URL parsedUrl = new URL(url);
        HttpURLConnection connection = openConnection(parsedUrl, request);
        for (String headerName : map.keySet()) {
            connection.addRequestProperty(headerName, map.get(headerName));
        }
        setConnectionParametersForRequest(connection, request);
        // Initialize HttpResponse with data from the HttpURLConnection.
        ProtocolVersion protocolVersion = new ProtocolVersion("HTTP", 1, 1);
        int responseCode = connection.getResponseCode();
        if (responseCode == -1) {
            // -1 is returned by getResponseCode() if the response code could not be retrieved.
            // Signal to the caller that something was wrong with the connection.
            throw new IOException("Could not retrieve response code from HttpUrlConnection.");
        }
        StatusLine responseStatus = new BasicStatusLine(protocolVersion,
                connection.getResponseCode(), connection.getResponseMessage());
        BasicHttpResponse response = new BasicHttpResponse(responseStatus);
        response.setEntity(entityFromConnection(connection));
        for (Entry<String, List<String>> header : connection.getHeaderFields().entrySet()) {
            if (header.getKey() != null) {
                Header h = new BasicHeader(header.getKey(), header.getValue().get(0));
                response.addHeader(h);
            }
        }
        return response;
    }

HttpClientStack：

 @Override
    public HttpResponse performRequest(Request<?> request, Map<String, String> additionalHeaders)
            throws IOException, AuthFailureError {
        HttpUriRequest httpRequest = createHttpRequest(request, additionalHeaders);
        addHeaders(httpRequest, additionalHeaders);
        addHeaders(httpRequest, request.getHeaders());
        onPrepareRequest(httpRequest);
        HttpParams httpParams = httpRequest.getParams();
        int timeoutMs = request.getTimeoutMs();
        // TODO: Reevaluate this connection timeout based on more wide-scale
        // data collection and possibly different for wifi vs. 3G.
        HttpConnectionParams.setConnectionTimeout(httpParams, 5000);
        HttpConnectionParams.setSoTimeout(httpParams, timeoutMs);
        return mClient.execute(httpRequest);
    }

两种不同的方式，但是返回都封装成了HttpResponse。

我们继续看执行网络请求的类Network和BasicNetwork
Network是一个接口，定义了一个执行网络请求的方法，BasicNetwork实现了这个接口，并根据不同的策略执行重连的操作。

/**
 * An interface for performing requests.
 */
public interface Network {
    /**
     * Performs the specified request.
     * @param request Request to process
     * @return A {@link NetworkResponse} with data and caching metadata; will never be null
     * @throws VolleyError on errors
     */
    public NetworkResponse performRequest(Request<?> request) throws VolleyError;
}

/**
 * A network performing Volley requests over an {@link HttpStack}.
 */
public class BasicNetwork implements Network {
    protected static final boolean DEBUG = VolleyLog.DEBUG;
    private static int SLOW_REQUEST_THRESHOLD_MS = 3000;
    private static int DEFAULT_POOL_SIZE = 4096;
    。。。
    @Override
    public NetworkResponse performRequest(Request<?> request) throws VolleyError {
        long requestStart = SystemClock.elapsedRealtime();
        while (true) {
            HttpResponse httpResponse = null;
            byte[] responseContents = null;
            Map<String, String> responseHeaders = new HashMap<String, String>();
            try {
                // 收集 headers.
                Map<String, String> headers = new HashMap<String, String>();
                addCacheHeaders(headers, request.getCacheEntry());
                //执行网络请求，获取返回数据
                httpResponse = mHttpStack.performRequest(request, headers);
                //获取状态码
                StatusLine statusLine = httpResponse.getStatusLine();
                int statusCode = statusLine.getStatusCode();
                //获取headers
                responseHeaders = convertHeaders(httpResponse.getAllHeaders());
                // 处理缓存验证.
                if (statusCode == HttpStatus.SC_NOT_MODIFIED) {
                    return new NetworkResponse(HttpStatus.SC_NOT_MODIFIED,
                            request.getCacheEntry() == null ? null : request.getCacheEntry().data,
                            responseHeaders, true);
                }
                // 处理变动资源，如重定向
                if (statusCode == HttpStatus.SC_MOVED_PERMANENTLY || statusCode == HttpStatus.SC_MOVED_TEMPORARILY) {
                    String newUrl = responseHeaders.get("Location");
                    request.setRedirectUrl(newUrl);
                }
                // 某些返回例如 204，不包含内容，需要检查  
                if (httpResponse.getEntity() != null) {
                  responseContents = entityToBytes(httpResponse.getEntity());
                } else {
                  // 对于无内容返回的请求，添加一个0字节的返回内容
                  responseContents = new byte[0];
                }
                // if the request is slow, log it.
                long requestLifetime = SystemClock.elapsedRealtime() - requestStart;
                logSlowRequests(requestLifetime, request, responseContents, statusLine);
                if (statusCode < 200 || statusCode > 299) {
                    throw new IOException();
                }
                return new NetworkResponse(statusCode, responseContents, responseHeaders, false);
            } catch (SocketTimeoutException e) {
                attemptRetryOnException("socket", request, new TimeoutError());
            } catch (ConnectTimeoutException e) {
                attemptRetryOnException("connection", request, new TimeoutError());
            } catch (MalformedURLException e) {
                throw new RuntimeException("Bad URL " + request.getUrl(), e);
            } catch (IOException e) {
                int statusCode = 0;
                NetworkResponse networkResponse = null;
                if (httpResponse != null) {
                    statusCode = httpResponse.getStatusLine().getStatusCode();
                } else {
                    throw new NoConnectionError(e);
                }
                if (statusCode == HttpStatus.SC_MOVED_PERMANENTLY || 
                        statusCode == HttpStatus.SC_MOVED_TEMPORARILY) {
                    VolleyLog.e("Request at %s has been redirected to %s", request.getOriginUrl(), request.getUrl());
                } else {
                    VolleyLog.e("Unexpected response code %d for %s", statusCode, request.getUrl());
                }
                if (responseContents != null) {
                    networkResponse = new NetworkResponse(statusCode, responseContents,
                            responseHeaders, false);
                    if (statusCode == HttpStatus.SC_UNAUTHORIZED ||
                            statusCode == HttpStatus.SC_FORBIDDEN) {
                        attemptRetryOnException("auth",
                                request, new AuthFailureError(networkResponse));
                    } else if (statusCode == HttpStatus.SC_MOVED_PERMANENTLY || 
                                statusCode == HttpStatus.SC_MOVED_TEMPORARILY) {
                        attemptRetryOnException("redirect",
                                request, new AuthFailureError(networkResponse));
                    } else {
                        // TODO: Only throw ServerError for 5xx status codes.
                        throw new ServerError(networkResponse);
                    }
                } else {
                    throw new NetworkError(networkResponse);
                }
            }
        }
    }
   。。。
}

Network对象生成后，会生成RequestQueue对象，看下RequestQueue的构造方法：

/**
     * Creates the worker pool. Processing will not begin until {@link #start()} is called.
     *
     * @param cache 一个存储responses到磁盘的缓存
     * @param network 一个Network接口用以执行HTTP requests
     * @param threadPoolSize network dispatcher线程数量
     * @param delivery 一个ResponseDelivery接口用以分发 responses and errors
     */
    public RequestQueue(Cache cache, Network network, int threadPoolSize,
            ResponseDelivery delivery) {
        mCache = cache;
        mNetwork = network;
        mDispatchers = new NetworkDispatcher[threadPoolSize];
        mDelivery = delivery;
    }

默认的构造方法线程数是4，会生成一个默认的ResponseDelivery。然后RequestQueue会执行一个start()方法，启动一个cache线程和多个网络任务线程，等待执行任务。看源码：

 /**
     * Starts the dispatchers in this queue.
     */
    public void start() {
        stop();  // Make sure any currently running dispatchers are stopped.
        // Create the cache dispatcher and start it.
        mCacheDispatcher = new CacheDispatcher(mCacheQueue, mNetworkQueue, mCache, mDelivery);
        mCacheDispatcher.start();
        // Create network dispatchers (and corresponding threads) up to the pool size.
        for (int i = 0; i < mDispatchers.length; i++) {
            NetworkDispatcher networkDispatcher = new NetworkDispatcher(mNetworkQueue, mNetwork,
                    mCache, mDelivery);
            mDispatchers[i] = networkDispatcher;
            networkDispatcher.start();
        }
    }

主体功能是创建一个内存分发器线程并启动，创建若干（默认是4）网络分发器线程并启动。分别来看下这两个类的代码，CacheDispatcher：
构造函数：

 /**
     * Creates a new cache triage dispatcher thread.  You must call {@link #start()}
     * in order to begin processing.
     *
     * @param cacheQueue 对传入请求进行分流的队列 
     * @param networkQueue 需要请求网络的requests 的队列
     * @param cache Cache interface to use for resolution
     * @param delivery Delivery 接口用以分发返回数据
     */
    public CacheDispatcher(
            BlockingQueue<Request<?>> cacheQueue, BlockingQueue<Request<?>> networkQueue,
            Cache cache, ResponseDelivery delivery) {
        mCacheQueue = cacheQueue;
        mNetworkQueue = networkQueue;
        mCache = cache;
        mDelivery = delivery;
    }

查看主要的run方法：

 @Override
    public void run() {
        if (DEBUG) VolleyLog.v("start new dispatcher");
        Process.setThreadPriority(Process.THREAD_PRIORITY_BACKGROUND);
        // Make a blocking call to initialize the cache.
        mCache.initialize();
        while (true) {
            try {
                // 这里是从mCacheQueue队列获取请求
                //注意这个队列的声明是BlockingQueue<Request<?>>
                //如果没有元素会是阻塞状态
                final Request<?> request = mCacheQueue.take();
                request.addMarker("cache-queue-take");
                // 检查请求是否已经被取消
                if (request.isCanceled()) {
                    request.finish("cache-discard-canceled");
                    continue;
                }
                // 尝试从缓存中去检索某一个请求
                Cache.Entry entry = mCache.get(request.getCacheKey());
                if (entry == null) {
                    request.addMarker("cache-miss");
                    // 缓存中没有，传入网络线程队列
                    mNetworkQueue.put(request);
                    continue;
                }
                // 如果已经过期，传入网络线程队列
                if (entry.isExpired()) {
                    request.addMarker("cache-hit-expired");
                    request.setCacheEntry(entry);
                    mNetworkQueue.put(request);
                    continue;
                }
                // 缓存区有次请求数据; 解析数据，回传给请求
                request.addMarker("cache-hit");
                Response<?> response = request.parseNetworkResponse(
                        new NetworkResponse(entry.data, entry.responseHeaders));
                request.addMarker("cache-hit-parsed");
                if (!entry.refreshNeeded()) {
                    // 没有过期，只需要回传数据即可
                    mDelivery.postResponse(request, response);
                } else {
                    // Soft-expired cache hit. We can deliver the cached response,
                    // but we need to also send the request to the network for
                    // refreshing.
                    request.addMarker("cache-hit-refresh-needed");
                    request.setCacheEntry(entry);
                    // Mark the response as intermediate.
                    response.intermediate = true;
                    // Post the intermediate response back to the user and have
                    // the delivery then forward the request along to the network.
                    mDelivery.postResponse(request, response, new Runnable() {
                        @Override
                        public void run() {
                            try {
                                mNetworkQueue.put(request);
                            } catch (InterruptedException e) {
                                // Not much we can do about this.
                            }
                        }
                    });
                }
            } catch (InterruptedException e) {
                // We may have been interrupted because it was time to quit.
                if (mQuit) {
                    return;
                }
                continue;
            }
        }
    }

线程权重设置为THREAD_PRIORITY_BACKGROUND，然后是不断的从缓存队列中取对象，缓存队列采用了阻塞队列，当队列中没有元素中时处于阻塞状态，一直到有新元素添加进来。当取到对象，查验是否被取消，然后去缓存中查找,如果存在，检测是否过期，否则都会扔进网络线程。最后有个refreshNeeded方法，这个涉及是否服务端有过期策略的支持，下边再说。

看看比较重要的NetworkDispatcher：
还是先看构造函数：

  /**
     * Creates a new network dispatcher thread.  You must call {@link #start()}
     * in order to begin processing.
     *
     * @param queue Queue of incoming requests for triage
     * @param network Network interface to use for performing requests
     * @param cache Cache interface to use for writing responses to cache
     * @param delivery Delivery interface to use for posting responses
     */
    public NetworkDispatcher(BlockingQueue<Request> queue,
            Network network, Cache cache,
            ResponseDelivery delivery) {
        mQueue = queue;
        mNetwork = network;
        mCache = cache;
        mDelivery = delivery;
    }

不同的是多了一个真正执行网络请求的network对象。

看看重要是run方法体：

@Override
    public void run() {
        Process.setThreadPriority(Process.THREAD_PRIORITY_BACKGROUND);
        Request request;
        while (true) {
            try {
                // 从队列中获取一个请求
                request = mQueue.take();
            } catch (InterruptedException e) {
                // We may have been interrupted because it was time to quit.
                if (mQuit) {
                    return;
                }
                continue;
            }
            try {
                request.addMarker("network-queue-take");
                // 检测是否已经被取消
                // 取消就不执行网络操作
                if (request.isCanceled()) {
                    request.finish("network-discard-cancelled");
                    continue;
                }
                // Tag the request (if API >= 14)
                if (Build.VERSION.SDK_INT >= 14) {
                    TrafficStats.setThreadStatsTag(request.getTrafficStatsTag());
                }
                // 执行网络请求，得到返回数据networkresponse
                NetworkResponse networkResponse = mNetwork.performRequest(request);
                request.addMarker("network-http-complete");
                // 如果服务端返回 304 我们已经分发过返回结果
                // 就此结束，不需要再发一遍结果
                if (networkResponse.notModified && request.hasHadResponseDelivered()) {
                    request.finish("not-modified");
                    continue;
                }
                // 在worker线程里解析返回的数据
                Response<?> response = request.parseNetworkResponse(networkResponse);
                request.addMarker("network-parse-complete");
                // 如果需要，写入缓存中
                // TODO: Only update cache metadata instead of entire record for 304s.
                if (request.shouldCache() && response.cacheEntry != null) {
                    mCache.put(request.getCacheKey(), response.cacheEntry);
                    request.addMarker("network-cache-written");
                }
                // 返回网络请求的结果
                request.markDelivered();
                mDelivery.postResponse(request, response);
            } catch (VolleyError volleyError) {
                parseAndDeliverNetworkError(request, volleyError);
            } catch (Exception e) {
                VolleyLog.e(e, "Unhandled exception %s", e.toString());
                mDelivery.postError(request, new VolleyError(e));
            }
        }
    }

网络请求的逻辑过程很清晰。这里我们再看看关于过期策略的问题。

   /** True if the entry is expired. */
        public boolean isExpired() {
            return this.ttl < System.currentTimeMillis();
        }
        /** True if a refresh is needed from the original data source. */
        public boolean refreshNeeded() {
            return this.softTtl < System.currentTimeMillis();
        }

这是判断是否过期，是否需要刷新的地方，其实就是用一个记录的时候看看是否超过当前时间。那就需要查查这个ttl，softttl时间戳是在哪设置的。跟进代码找到设置的地方。在HttpHeaderParser类中的parseCacheHeaders方法里会对网络返回的数据进行解析，返回一个Cache.Entry对象。这里会设置时间戳。具体代码如下：

headerValue = headers.get("Cache-Control");
        if (headerValue != null) {
            hasCacheControl = true;
            String[] tokens = headerValue.split(",");
            for (int i = 0; i < tokens.length; i++) {
                String token = tokens[i].trim();
                if (token.equals("no-cache") || token.equals("no-store")) {
                    return null;
                } else if (token.startsWith("max-age=")) {
                    try {
                        maxAge = Long.parseLong(token.substring(8));
                    } catch (Exception e) {
                    }
                } else if (token.equals("must-revalidate") || token.equals("proxy-revalidate")) {
                    maxAge = 0;
                }
            }
        }
 headerValue = headers.get("Expires");
        if (headerValue != null) {
            serverExpires = parseDateAsEpoch(headerValue);
        }
        serverEtag = headers.get("ETag");
        // Cache-Control takes precedence over an Expires header, even if both exist and Expires
        // is more restrictive.
        if (hasCacheControl) {
            softExpire = now + maxAge * 1000;
        } else if (serverDate > 0 && serverExpires >= serverDate) {
            // Default semantic for Expire header in HTTP specification is softExpire.
            softExpire = now + (serverExpires - serverDate);
        }
        Cache.Entry entry = new Cache.Entry();
        entry.data = response.data;
        entry.etag = serverEtag;
        entry.softTtl = softExpire;
        entry.ttl = entry.softTtl;
        entry.serverDate = serverDate;
        entry.responseHeaders = headers;

从返回的header中取serverExpires，Cache-Control比Expires header具有更高的优先级，如果返回的header中包含Cache-Control，那就从header中获取maxAge，softExpire就是maxAge秒的有效期。否则就是从header中获取serverExpires和serverDate，两者相减就是有效期的数值。缓存中保存的对象有效期都被设置成了softExpire。

最后附上整体流程图：

参考资料：
http://tomkeyzhang.duapp.com/?p=7
Volley源码：
https://github.com/mcxiaoke/android-volley
作者：flyouting