ThreadPoolExecutor线程池源码分析

时间 2019-12-05

标签 threadpoolexecutor 线程源码分析繁體版

原文原文链接

了解ThreadPoolExecutor

先看一下线程池类的类图关系：java

Executor接口

Executor做者描述的是Executor提供了一种解耦方式将任务的提交和任务以何种技术执行分离；
Executor接口只有一个方法：安全

void execute(Runnable command);

execute方法接收一个Runnable对象，方法的描述是在将来的某个时间执行command。不论是在一个新的线程中执行，仍是在线程池中执行，甚至在调用者线程中当即执行。异步

ExecutorService接口

ExecutorService继承了Executor接口，ExecutorService能够被关闭，关闭之后再也不接收新的任务。ExecutorService提供了两个不一样的方法关闭ExecutorService。shutdown方法会等待以前还未执行的任务执行完毕再关闭，而shutdownNow则不会再启动新的任务，还会中断正在执行的任务。一旦关闭后，ExecutorService就不会有正在执行的任务，也不会有等待被执行的任务，更不会有新的任务被提交。ExecutorService关闭后应该处理好一些资源的回收。ide

ThreadPoolExecutor

线程池技术旨在解决两个不一样的问题：函数

在处理大量异步任务时能够提升性能，由于减小了线程的销毁，新建，切换等消耗性能的操做。
线程池还有能力统一管理，调度，监控，调优线程等，还提供了一下基本的统计，好比已完成的任务数量。

重要的状态和状态判断的方法

private final AtomicInteger ctl = new AtomicInteger(ctlOf(RUNNING, 0));
//高3位和低29位分别表示状态和线程数
private static final int COUNT_BITS = Integer.SIZE - 3;
//1左移29位减一获得低29位都是1，即线程的最大数量，大概5亿多
private static final int CAPACITY   = (1 << COUNT_BITS) - 1;

// runState is stored in the high-order bits
private static final int RUNNING    = -1 << COUNT_BITS;//111
private static final int SHUTDOWN   =  0 << COUNT_BITS;//000
private static final int STOP       =  1 << COUNT_BITS;//001
private static final int TIDYING    =  2 << COUNT_BITS;//010
private static final int TERMINATED =  3 << COUNT_BITS;//011

// Packing and unpacking ctl
//得到状态
private static int runStateOf(int c)     { return c & ~CAPACITY; }
//得到线程数量
private static int workerCountOf(int c)  { return c & CAPACITY; }
//经过状态和线程数量组装ctl
private static int ctlOf(int rs, int wc) { return rs | wc; }

/*
 * Bit field accessors that don't require unpacking ctl.
 * These depend on the bit layout and on workerCount being never negative.
 */
//c状态是否小于s状态
private static boolean runStateLessThan(int c, int s) {
    return c < s;
}
//c状态是否大于等于s状态
private static boolean runStateAtLeast(int c, int s) {
    return c >= s;
}
//线程池是不是运行状态
private static boolean isRunning(int c) {
    return c < SHUTDOWN;
}

整个类最重要的一个状态标志ctl是一个AtomicInteger，它包含了两个字段的含义。workerCount线程数量，runState线程池的状态。
这一个字段是如何包含两个字段的含义的呢，Doug Lea大牛使用了一个int的32位bits的高三位保存了状态值，低29位保存了线程数量。oop

其中五个状态：
RUNNING：接收新的任务，处理队列中的任务；
SHUTDOWN：不接收新的任务，但处理队列中的任务；
STOP：不接收新的任务，不处理队列中的任务，中断正在执行的任务；
TIDYING：全部任务都终止，线程数为0，线程过分到TIDYING时会调用terminated钩子方法；
TERMINATED：terminated执行完毕；性能

状态之间的转换：
RUNNING -> SHUTDOWN：调用shutdown方法；
(RUNNING or SHUTDOWN) -> STOP：调用shutdownNow方法；
SHUTDOWN -> TIDYING：当线程池和任务队列都为空；
STOP -> TIDYING：当线程池为空；
TIDYING -> TERMINATED：当terminated方法执行完毕；ui

Worker介绍

Worker类主要包含了线程运行任务时的终端控制状态，同时还有一些少许的信息记录。Worker适时的继承了AQS，让线程在任务执行之间获取锁和释放锁变得简单。这确保了中断是唤醒一个等待任务的线程，而不是中断一个正在运行的任务线程。this

private final class Worker
    extends AbstractQueuedSynchronizer
    implements Runnable
{
    /**
     * This class will never be serialized, but we provide a
     * serialVersionUID to suppress a javac warning.
     */
    private static final long serialVersionUID = 6138294804551838833L;

    /** Thread this worker is running in.  Null if factory fails. */
    final Thread thread;
    /** Initial task to run.  Possibly null. */
    Runnable firstTask;
    /** Per-thread task counter */
    volatile long completedTasks;

    /**
     * Creates with given first task and thread from ThreadFactory.
     * @param firstTask the first task (null if none)
     */
    Worker(Runnable firstTask) {
        setState(-1); // inhibit interrupts until runWorker
        this.firstTask = firstTask;
        this.thread = getThreadFactory().newThread(this);
    }

    /** Delegates main run loop to outer runWorker  */
    public void run() {
        runWorker(this);
    }

    // Lock methods
    //
    // The value 0 represents the unlocked state.
    // The value 1 represents the locked state.

    protected boolean isHeldExclusively() {
        return getState() != 0;
    }

    protected boolean tryAcquire(int unused) {
        if (compareAndSetState(0, 1)) {
            setExclusiveOwnerThread(Thread.currentThread());
            return true;
        }
        return false;
    }

    protected boolean tryRelease(int unused) {
        setExclusiveOwnerThread(null);
        setState(0);
        return true;
    }

    public void lock()        { acquire(1); }
    public boolean tryLock()  { return tryAcquire(1); }
    public void unlock()      { release(1); }
    public boolean isLocked() { return isHeldExclusively(); }

    void interruptIfStarted() {
        Thread t;
        if (getState() >= 0 && (t = thread) != null && !t.isInterrupted()) {
            try {
                t.interrupt();
            } catch (SecurityException ignore) {
            }
        }
    }
}

Worker继承了AQS，实现了Runnable接口；在构造函数中，初始化了它的第一次仍无，使用threadFactory建立一个新的线程；
Worker继承AQS，目的是想使用独占锁来表示线程是否正在执行任务，Worker的线程获取了独占锁就说明它在执行任务，不能被中断。从tryAcquire方法能够看出，它实现的是不可重入锁，由于是否得到锁在这里表示一个状态，若是能够重入的话，独占锁就失去了只表示一个状态的含义。在构造函数初始化时，Worker将state设置为-1，由于在tryAcquire中CAS操做compareAndSetState(0, 1)，表示state在-1时不能被中断。在runWorker中将state设置为0.atom

ThreadPooleExecutor构造方法

public ThreadPoolExecutor(int corePoolSize,
                          int maximumPoolSize,
                          long keepAliveTime,
                          TimeUnit unit,
                          BlockingQueue<Runnable> workQueue,
                          ThreadFactory threadFactory,
                          RejectedExecutionHandler handler) {
    if (corePoolSize < 0 ||
        maximumPoolSize <= 0 ||
        maximumPoolSize < corePoolSize ||
        keepAliveTime < 0)
        throw new IllegalArgumentException();
    if (workQueue == null || threadFactory == null || handler == null)
        throw new NullPointerException();
    this.corePoolSize = corePoolSize;
    this.maximumPoolSize = maximumPoolSize;
    this.workQueue = workQueue;
    this.keepAliveTime = unit.toNanos(keepAliveTime);
    this.threadFactory = threadFactory;
    this.handler = handler;
}

说明一下各参数的含义：
corePoolSize：核心线程数量，即便线程是空闲的也保持在线程池中，除非allowCoreThreadTimeOut参数被设置；

maximumPoolSize：最大线程数量；

keepAliveTime：当线程数量超过核心线程数量时，超出的空闲线程等待新任务的最大时长；

unit：时间单位；

workQueue：存放将要被执行的任务的队列；

threadFactory：建立线程的线程工厂；

handler：当任务队列满且没有空闲的线程时处理任务的handler，线程池提供了四种策略：

AbortPolicy：直接抛出异常，默认；
CallerRunsPolicy：使用调用者的线程执行；
DiscardOldestPolicy：抛弃队列最前的任务，执行当前任务；
DiscardPolicy：直接丢弃任务；

这些参数对整个线程池运行很是重要；

execute方法

public void execute(Runnable command) {
    if (command == null)
        throw new NullPointerException();
    /*
     * Proceed in 3 steps:
     *
     * 1. If fewer than corePoolSize threads are running, try to
     * start a new thread with the given command as its first
     * task.  The call to addWorker atomically checks runState and
     * workerCount, and so prevents false alarms that would add
     * threads when it shouldn't, by returning false.
     *
     * 2. If a task can be successfully queued, then we still need
     * to double-check whether we should have added a thread
     * (because existing ones died since last checking) or that
     * the pool shut down since entry into this method. So we
     * recheck state and if necessary roll back the enqueuing if
     * stopped, or start a new thread if there are none.
     *
     * 3. If we cannot queue task, then we try to add a new
     * thread.  If it fails, we know we are shut down or saturated
     * and so reject the task.
     */
    //获取ctl
    int c = ctl.get();
    //若是线程数小于核心线程数
    if (workerCountOf(c) < corePoolSize) {
        //添加线程并执行任务
        if (addWorker(command, true))
            return;
        c = ctl.get();
    }
    //线程数大于核心线程数
    //若是线程池running状态且添加任务到队列成功
    if (isRunning(c) && workQueue.offer(command)) {
        int recheck = ctl.get();
        //若是线程池不是运行状态，队列移除任务，使用拒绝策略处理任务
        if (! isRunning(recheck) && remove(command))
            reject(command);
        //若是这时线程数为0，添加任务
        else if (workerCountOf(recheck) == 0)
            addWorker(null, false);
    }
    //队列满，添加线程失败，使用拒绝策略处理任务
    else if (!addWorker(command, false))
        reject(command);
}

在线程池添

数量若是小于核心线程数，则添加新的线程并执行当前任务，不然判断若是队列是否未满，则添加当前任务到队列，不然判断线程数量若是小于最大线程数，则添加新的线程并执行，不然使用拒绝策略处理当前任务。

addWorker方法

addWorker方法主要是添加线程并执行任务：

private boolean addWorker(Runnable firstTask, boolean core) {
    retry:
    for (;;) {
        int c = ctl.get();
        //获取线程池运行状态
        int rs = runStateOf(c);

        // Check if queue empty only if necessary.
        //若是运行状态大于等于SHUTDOWN，再也不接受新的任务，返回false
        //若是运行状态等于SHUTDOWN且firstTask不为空，继续执行下去，若是firstTask为空，queue为空，返回false，不然继续执行；只要SHUTDOWN状态下还有任务在，就须要往下执行，可能须要新建worker执行
        if (rs >= SHUTDOWN &&
            ! (rs == SHUTDOWN &&
               firstTask == null &&
               ! workQueue.isEmpty()))
            return false;

        for (;;) {
            //得到线程数量
            int wc = workerCountOf(c);
            //若是线程数量大于容量或者当core为true时wc大于等于核心线程数，当core为falsewc大于等于最大线程数量时，返回false
            if (wc >= CAPACITY ||
                wc >= (core ? corePoolSize : maximumPoolSize))
                return false;
            //CAS线程数加一，成功则中断循环
            if (compareAndIncrementWorkerCount(c))
                break retry;
            //若是CAS失败，从新获取ctl，线程池运行状态没变的话继续loop
            c = ctl.get();  // Re-read ctl
            if (runStateOf(c) != rs)
                continue retry;
            // else CAS failed due to workerCount change; retry inner loop
        }
    }

    boolean workerStarted = false;
    boolean workerAdded = false;
    Worker w = null;
    try {
        //新建一个worker
        w = new Worker(firstTask);
        //能获得worker的thread
        final Thread t = w.thread;
        if (t != null) {
            final ReentrantLock mainLock = this.mainLock;
            mainLock.lock();
            try {
                // Recheck while holding lock.
                // Back out on ThreadFactory failure or if
                // shut down before lock acquired.
                int rs = runStateOf(ctl.get());

                //若是rs是RUNNING或者SHUTDOWN且firstTask为null
                //由于SHUTDOWN时还须要执行queue中的任务
                if (rs < SHUTDOWN ||
                    (rs == SHUTDOWN && firstTask == null)) {
                    if (t.isAlive()) // precheck that t is startable
                        throw new IllegalThreadStateException();
                    //往线程池中添加worker
                    workers.add(w);
                    int s = workers.size();
                    //记录线程池出现的最大线程数量
                    if (s > largestPoolSize)
                        largestPoolSize = s;
                    workerAdded = true;
                }
            } finally {
                mainLock.unlock();
            }
            if (workerAdded) {
                //启动worker
                t.start();
                workerStarted = true;
            }
        }
    } finally {
        if (! workerStarted)
            addWorkerFailed(w);
    }
    return workerStarted;
}

worker的run方法调用的是runWorker；

runWorker方法

final void runWorker(Worker w) {
    Thread wt = Thread.currentThread();
    //保存worker的第一个任务
    Runnable task = w.firstTask;
    //清空worker的第一个任务
    w.firstTask = null;
    //这里将worker的state设置为0，容许中断
    w.unlock(); // allow interrupts
    boolean completedAbruptly = true;
    try {
        //若是task为空，则从队列中获取任务
        while (task != null || (task = getTask()) != null) {
            //开始执行任务，不容许中断
            w.lock();
            // If pool is stopping, ensure thread is interrupted;
            // if not, ensure thread is not interrupted.  This
            // requires a recheck in second case to deal with
            // shutdownNow race while clearing interrupt
            //若是当前状态大于等于STOP要保持当前线程中断
            //若是当前线程小于STOP即RUNNING或者SHUTDOWN，调用Thread.interrupted()清空中断标志，若是这时调用了shutdownNow状态为STOP，仍是要保持中断状态
            if ((runStateAtLeast(ctl.get(), STOP) ||
                 (Thread.interrupted() &&
                  runStateAtLeast(ctl.get(), STOP))) &&
                !wt.isInterrupted())
                wt.interrupt();
            try {
                //执行任务前作的事
                beforeExecute(wt, task);
                Throwable thrown = null;
                try {
                    //执行任务
                    task.run();
                } catch (RuntimeException x) {
                    thrown = x; throw x;
                } catch (Error x) {
                    thrown = x; throw x;
                } catch (Throwable x) {
                    thrown = x; throw new Error(x);
                } finally {
                    //执行任务以后作的事
                    afterExecute(task, thrown);
                }
            } finally {
                task = null;
                //worker的完成任务数量加一，此时是线程安全的
                w.completedTasks++;
                //释放锁
                w.unlock();
            }
        }
        completedAbruptly = false;
    } finally {
        //线程退出
        processWorkerExit(w, completedAbruptly);
    }
}

每一个task在调用runWorker后会一直循环执行任务，直到queue中没有任务了，循环结束，worker生命周期结束。

getTask

上面runWorker时调用了getTask去获取队列中的任务，下面咱们看一下这个方法：

private Runnable getTask() {
    boolean timedOut = false; // Did the last poll() time out?

    for (;;) {
        int c = ctl.get();
        int rs = runStateOf(c);

        // Check if queue empty only if necessary.
        //若是rs大于等于SHUTDOWN，当RS大于等于STOP说明线程池已经不处理队列中的任务了，当rs为SHUTDOWN时，若是队列是空的，返回null
        if (rs >= SHUTDOWN && (rs >= STOP || workQueue.isEmpty())) {
            //线程数减一
            decrementWorkerCount();
            return null;
        }

        int wc = workerCountOf(c);

        // Are workers subject to culling?
        //是否超时控制，allowCoreThreadTimeOut默认false，表明不容许核心线程超时，对于超出核心线程的线程须要控制超时
        boolean timed = allowCoreThreadTimeOut || wc > corePoolSize;
        //当线程数大于最大线程数，或者须要超时控制且上次获取任务超时
        //且线程数大于1或者队列为空，尝试将线程数减一并返回null
        if ((wc > maximumPoolSize || (timed && timedOut))
            && (wc > 1 || workQueue.isEmpty())) {
            if (compareAndDecrementWorkerCount(c))
                return null;
            //失败重试
            continue;
        }

        try {
            //当须要超时控制时，在keepAliveTime时间内没有获取到任务的话返回null，不然调用take获取任务，此时线程时阻塞的
            Runnable r = timed ?
                workQueue.poll(keepAliveTime, TimeUnit.NANOSECONDS) :
                workQueue.take();
            if (r != null)
                return r;
            timedOut = true;
        } catch (InterruptedException retry) {
            timedOut = false;
        }
    }
}

getTask方法在线程数量大于核心线程数时会判断在获取task时进行超时判断（poll），超时返回null这时getTask返回null，那当前worker的loop结束即run方法结束，线程生命周期结束。而核心线程则会调用take方法，当没有任务时会阻塞。

processWorkerExit

runTask方法最后会调用processWorkerExit方法进行一些cleanup工做。

private void processWorkerExit(Worker w, boolean completedAbruptly) {
    //completedAbruptly为true时表明发生了异常，线程数减一
    if (completedAbruptly) // If abrupt, then workerCount wasn't adjusted
        decrementWorkerCount();

    final ReentrantLock mainLock = this.mainLock;
    mainLock.lock();
    try {
        //统计完成任务数
        completedTaskCount += w.completedTasks;
        //线程池移除当前worker
        workers.remove(w);
    } finally {
        mainLock.unlock();
    }
    // 根据线程池状态进行判断是否结束线程池
    tryTerminate();

    int c = ctl.get();
    //当线程池状态为RUNNING或者SHUTDOWN时
    //若是发生异常，从新加入一个worker replacement
    if (runStateLessThan(c, STOP)) {
        if (!completedAbruptly) {
            //当allowCoreThreadTimeOut为true，最少要一个worker
            int min = allowCoreThreadTimeOut ? 0 : corePoolSize;
            if (min == 0 && ! workQueue.isEmpty())
                min = 1;
            //当线程数大于等于最少须要的线程数，则不须要add新的worker
            if (workerCountOf(c) >= min)
                return; // replacement not needed
        }
        addWorker(null, false);
    }
}

tryTerminate方法

上面咱们跳过了tryTerminate方法，该方法判断是否要结束线程池，这里看一下

final void tryTerminate() {
    for (;;) {
        int c = ctl.get();
        //当线程池状态时RUNNING或者已经TIDYING或者已经TERMINATED或者SHUTDOWN且还有任务没有被执行，直接返回
        if (isRunning(c) ||
            runStateAtLeast(c, TIDYING) ||
            (runStateOf(c) == SHUTDOWN && ! workQueue.isEmpty()))
            return;
        // 若是线程数不为0，则中断一个空闲的工做线程
        if (workerCountOf(c) != 0) { // Eligible to terminate
            //workQueue.take()时若是queue一直为空的话，线程会一直阻塞
            interruptIdleWorkers(ONLY_ONE);
            return;
        }

        final ReentrantLock mainLock = this.mainLock;
        mainLock.lock();
        try {
            //若是状态设置成功为TIDYING，调用勾子方法terminated，该方法留给了子类实现
            if (ctl.compareAndSet(c, ctlOf(TIDYING, 0))) {
                try {
                    terminated();
                } finally {
                    //设置状态为TERMINATED
                    ctl.set(ctlOf(TERMINATED, 0));
                    termination.signalAll();
                }
                return;
            }
        } finally {
            mainLock.unlock();
        }
        // else retry on failed CAS
    }
}

interruptIdleWorkers

上面说为了当队列一直为空的时候，核心线程会一直阻塞，因此调用了interruptIdleWorkers，咱们看一下执行了什么：

private void interruptIdleWorkers(boolean onlyOne) {
    final ReentrantLock mainLock = this.mainLock;
    mainLock.lock();
    try {
        for (Worker w : workers) {
            Thread t = w.thread;
            if (!t.isInterrupted() && w.tryLock()) {
                try {
                    t.interrupt();
                } catch (SecurityException ignore) {
                } finally {
                    w.unlock();
                }
            }
            if (onlyOne)
                break;
        }
    } finally {
        mainLock.unlock();
    }
}

遍历线程池中全部的线程，若线程没有被中断tryLock成功，就中断该线程，LockSupport.park()能响应中断信号，阻塞的线程被中断唤醒。