Underwater imaging with red-NIR light illumination can avoid phototropic aggregation-induced observational deviation of marine plankton abundance under white light illumination, but this will lead to the loss of critical color information in the collected grayscale images, which is non-preferable to subsequent human and machine recognition. We present a novel deep networks-based vision system IsPlanktonCLR for automatic colorization of in situ marine plankton images. IsPlanktonCLR uses a reference module to generate self-guidance from a customized palette, which is obtained by clustering in situ plankton image colors. With this self-guidance, a parallel colorization module restores input grayscale images into their true color counterparts. Additionally, a new metric for image colorization evaluation is proposed, which can objectively reflect the color dissimilarity between comparative images. Experiments and comparisons with state-of-the-art approaches are presented to show that our method achieves a substantial improvement over previous methods on color restoration of scientific plankton image data.