Overview of Machine Learning: Part 2

Deep Learning for Medical Image Analysis
  • Author Footnotes
    1 Co-first author.
    William Trung Le
    Footnotes
    1 Co-first author.
    Affiliations
    Polytechnique Montreal, PO Box 6079, succ. Centre-ville, Montreal, Quebec H3C 3A7, Canada

    CHUM Research Center, 900 St Denis Street, Montreal, Quebec H2X 0A9, Canada
    Search for articles by this author
  • Author Footnotes
    1 Co-first author.
    Farhad Maleki
    Footnotes
    1 Co-first author.
    Affiliations
    Augmented Intelligence & Precision Health Laboratory (AIPHL), Department of Radiology and Research Institute of the McGill University Health Centre, 1001 Decarie Boulevard, Montreal, Quebec H4A 3J1, Canada
    Search for articles by this author
  • Francisco Perdigón Romero
    Affiliations
    Polytechnique Montreal, PO Box 6079, succ. Centre-ville, Montreal, Quebec H3C 3A7, Canada
    Search for articles by this author
  • Reza Forghani
    Affiliations
    Augmented Intelligence & Precision Health Laboratory (AIPHL), Department of Radiology and Research Institute of the McGill University Health Centre, 1001 Decarie Boulevard, Montreal, Quebec H4A 3J1, Canada

    Segal Cancer Centre, Lady Davis Institute for Medical Research, Jewish General Hospital, 3755 Cote Ste-Catherine Road, Montreal, Quebec H3T 1E2, Canada

    Gerald Bronfman Department of Oncology, McGill University, Montreal, Quebec, Canada

    Department of Otolaryngology - Head and Neck Surgery, McGill University, Montreal, Quebec, Canada
    Search for articles by this author
  • Samuel Kadoury
    Correspondence
    Corresponding author.
    Affiliations
    Polytechnique Montreal, PO Box 6079, succ. Centre-ville, Montreal, Quebec H3C 3A7, Canada

    CHUM Research Center, 900 St Denis Street, Montreal, Quebec H2X 0A9, Canada
    Search for articles by this author
  • Author Footnotes
    1 Co-first author.
Published:September 18, 2020DOI:https://doi.org/10.1016/j.nic.2020.06.003

      Keywords

      To read this article in full you will need to make a payment
      Purchase one-time access
      Subscribers receive full online access to your subscription and archive of back issues up to and including 2002.
      Content published before 2002 is available via pay-per-view purchase only.
      Subscribe to Neuroimaging Clinics
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect

      References

        • Silver D.
        • Schrittwieser J.
        • Simonyan K.
        • et al.
        Mastering the game of Go without human knowledge.
        Nature. 2017; 550: 354-359
        • Vinyals O.
        • Babuschkin I.
        • Czarnecki W.M.
        • et al.
        Grandmaster level in StarCraft Ii using multi-agent reinforcement learning.
        Nature. 2019; 575: 350-354
      1. Fridman L, Brown DE, Glazer M, et al. MIT autonomous vehicle technology study: Large-scale deep learning based analysis of driver behavior and interaction with automation. arXiv preprint arXiv:1711.06976 1 (2017).

        • Lin E.D.
        • Hefner J.L.
        • Zeng X.
        • et al.
        A deep learning model for pediatric patient risk stratification.
        Am J Manag Care. 2019; 25: e310-e315
        • McKinney S.M.
        • Sieniek M.
        • Godbole V.
        • et al.
        International evaluation of an AI system for breast cancer screening.
        Nature. 2020; 577: 89-94
        • Vaswani A.
        • Shazeer N.
        • Parmar N.
        • et al.
        Attention is all you need.
        Adv Neural Inf Process Syst. 2017; : 5998-6008
      2. Devlin J, Chang M.-W, Lee K, et al. BERT: pre-training of deep bidirectional transformers for language understanding, arXiv preprint arXiv:1810.04805.

        • Goodfellow I.
        • Bengio Y.
        • Courville A.
        Deep learning.
        MIT Press, Cambridge2016
      3. Abadi M, Barham P, Chen, J, et al. Tensorflow: A system for large-scale machine learning. In 12th {USENIX} symposium on operating systems design and implementation ({OSDI} 16). Savanah (GA); 2016. p. 265–83.

        • Chollet F.
        Keras.
        (Available at:) (Accessed July 29, 2020)
        • Paske A.
        • Gross S.
        • Massa F.
        • et al.
        PyTorch: An imperative style, high-performance deep learning library.
        Adv Neural Inf Process Syst. 2019; 32: 8024-8035
      4. Chen T, Li M, Li Y, et al. MXNet: A flexible and efficient machine learning library for heterogeneous distributed systems, arXiv preprint arXiv:1512.01274.

      5. JiaY, Shelhamer E, Donahue J, et al. Caffe: Convolutional architecture for fast feature embedding, arXiv preprint arXiv:1408.5093.

        • Manyika J.
        • Chui M.
        • Brown B.
        • et al.
        Big data: the next frontier for innovation, competition, and productivity, Tech. rep..
        McKinsey Global Institute, New York2011
        • McCulloch W.S.
        • Pitts W.
        A logical calculus of the ideas immanent in nervous activity.
        Bull Math Biophys. 1943; 5: 115-133
        • Rosenblatt F.
        The perceptron: A probabilistic model for information storage and organization in the brain.
        Psychol Rev. 1958; 65: 386-408
        • Rumelhart D.E.
        • Hinton G.E.
        • Williams R.J.
        Learning representations by back-propagating errors.
        Nature. 1986; 323: 533-536
      6. Deng J, Dong W, Socher R, et al. Imagenet: A large-scale hierarchical image database. IEEE Conference on computer vision and Pattern Recognition, IEEE. Miami (FL), June 20, 2009.

        • Krizhevsky A.
        • Sutskever I.
        • Hinton G.E.
        Imagenet classification with deep convolutional neural networks.
        in: Pereira F. Burges C.J.C. Bottou L. Advances in neural information processing systems. Curran Associates, Inc, Denver (CO)2012: 1097-1105
      7. Glorot X, Bordes A, Bengio Y. Deep sparse rectifier neural networks. Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics. Fort Lauderdale (FL); 2011. p. 315–23.

      8. Maas AL, Hannun AY, Ng AY. Rectifier nonlinearities improve neural network acoustic models. Proceedings of the 30th Annual International Conference on Machine Learning, Vol. 30. Atlanta (GA); 2013. p. 3.

      9. Rifai S, Vincent P, Muller X, et al. Contractive autoencoders: explicit invariance during feature extraction. Proceedings of the 28th International Conference on International Conference on Machine Learning. Bellevue (WA): ACM; 2011. p. 833–840.

      10. Rifai S, Mesnil G, Vincent P, et al. Higher order contractive auto-encoder. Joint European Conference on Machine Learning and Knowledge Discovery in Databases. Athens (Greece): Springer; 2011. p. 645–60.

      11. Vincent P, Larochelle H, Bengio Y, et al. Extracting and composing robust features with denoising autoencoders. Proceedings of the 25th International Conference on Machine Learning. Helsinki (Finland): ACM; 2008. p. 1096– 1103.

      12. Kingma DP, Welling M. Auto-encoding variational bayes, arXiv preprint arXiv:1312.6114.

        • Vincent P.
        • Larochelle H.
        • Lajoie I.
        • et al.
        Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion.
        J Mach Learn Res. 2010; 11: 3371-3408
        • Goodfellow I.
        • Pouget-Abadie J.
        • Mirza M.
        • et al.
        Generative adversarial nets.
        Adv Neural Inf Process Syst. 2014; 27: 2672-2680
        • Fan J.
        • Cao X.
        • Xue Z.
        • et al.
        Adversarial similarity network for evaluating image alignment in deep learning based registration.
        in: Frangi A. Schnabel J. Davatzikos C. Medical image computing and computer Assisted Intervention – MICCAI 2018. Springer International Publishing, Cham (Switzerland)2018: 739-746
      13. Mahapatra D, Antony B, Sedai S, et al. Deformable medical image registration using generative adversarial networks. 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018). Washington, DC: IEEE; 2018. p. 1449–1453.

      14. Kazeminia S, Baur C, Kuijper A, et al. GANs for medical image analysis, arXiv preprint arXiv:1809.06222.

        • Bi L.
        • Kim J.
        • Kumar A.
        • et al.
        Synthesis of positron emission tomography (PET) images via multi-channel generative adversarial networks (GANs).
        in: Frangi A. Schnabel J. Davatzikos C. Molecular imaging, reconstruction and analysis of Moving body organs, and stroke imaging and treatment. Springer International Publishing, Cham (Switzerland)2017: 43-51
      15. Yi X, Walia E, Babyn P. Generative adversarial network in medical imaging: A review. Med Image Anal 2019. https://doi.org/10.1016/j.media.2019.101552.

        • Nie D.
        • Trullo R.
        • Lian J.
        • et al.
        Medical image synthesis with deep convolutional adversarial networks.
        IEEE Trans Biomed Eng. 2018; 65: 2720-2730
        • Frid-Adar M.
        • Diamant I.
        • Klang E.
        • et al.
        GAN-based synthetic medical image augmentation for increased CNN performance in liver lesion classification.
        Neurocomputing. 2018; 321: 321-331
        • Nie D.
        • Trullo R.
        • Lian J.
        • et al.
        Medical image synthesis with context-aware generative adversarial networks.
        in: Frangi A. Schnabel J. Davatzikos C. Medical image computing and computer Assisted Intervention - MICCAI 2017. Springer International Publishing, Cham (Switzerland)2017: 417-425
        • Cerrolaza J.J.
        • Li Y.
        • Biffi C.
        • et al.
        3D fetal skull reconstruction from 2DUS via deep conditional generative networks.
        in: Frangi A. Schnabel J. Davatzikos C. Medical image computing and computer Assisted Intervention – MICCAI 2018. Springer International Publishing, 2018: 383-391
      16. Biffi C, Cerrolaza JJ, Tarroni G, et al. 3d high-resolution cardiac segmentation reconstruction from 2d views using conditional variational autoencoders, arXiv preprint arXiv:1902.11000.

        • Schmidhuber J.
        • Hochreiter S.
        Long short-term memory.
        Neural Comput. 1997; 9: 1735-1780
      17. Cho K, Van Merrïenboer B, Bahdanau D, et al. On the properties of neural machine translation: Encoder-decoder approaches, arXiv preprint arXiv:1409.1259.

      18. Chung J, Gulcehre C, Cho K, et al. Empirical evaluation of gated recurrent neural networks on sequence modeling. NIPS 2014 Workshop on Deep Learning. Montreal (QC), December 8, 2014.

        • Weiss G.
        • Goldberg Y.
        • Yahav E.
        On the practical computational power of finite precision rnns for language recognition. Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers).
        Association for Computational Linguistics, Melbourne (Australia)2018: 740-745
        • Glorot X.
        • Bengio Y.
        Understanding the difficulty of training deep feedforward neural networks. Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics.
        PMLR, Sardinia (Italy)2010: 249-256
        • He K.
        • Zhang X.
        • Ren S.
        • et al.
        Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. Proceedings of the IEEE International Conference on Computer Vision.
        IEEE, Las Condes (Chile)2015: 1026-1034
        • Srivastava N.
        • Hinton G.
        • Krizhevsky A.
        • et al.
        Dropout: a simple way to prevent neural networks from overfitting.
        J Mach Learn Res. 2014; 15: 1929-1958
        • Shorten C.
        • Khoshgoftaar T.M.
        A survey on image data augmentation for deep learning.
        J Big Data. 2019; 6: 60
        • Nalepa J.
        • Marcinkiewicz M.
        • Kawulok M.
        Data augmentation for braintumor segmentation: A review.
        Front Comput Neurosci. 2019; 13: 83
        • Branco P.
        • Torgo L.i.
        • Ribeiro R.
        A survey of predictive modeling on imbalanced domains.
        ACM Comput Surv. 2016; 49: 1-31
        • Yang Q.
        • Zhang Y.
        • Dai W.
        • et al.
        Transfer learning.
        Cambridge University Press, Cambridge2020
        • Peng J.
        • Kang S.
        • Ning Z.
        • et al.
        Residual convolutional neural network for predicting response of transarterial chemoembolization in hepatocellular carcinoma from CT imaging.
        Eur Radiol. 2019; 30: 413-424
      19. Tousignant A, Lemaˆıtre P, Precup D, et al. Prediction of disease progression in multiple sclerosis patients using deep learning analysis of MRI data. Proceedings of The 2nd International Conference on Medical Imaging with Deep Learning, Vol. 102. London (UK): MIDL; 2019. p. 483–92.

        • Zhou L.
        • Zhang Z.
        • Chen Y.-C.
        • et al.
        A deep learning-based radiomics model for differentiating benign and malignant renal tumors.
        Translational Oncol. 2019; 12: 292-300
      20. He K, Zhang X, Ren S, et al. Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas (NV): IEEE; 2016. p. 770–8.

        • Szegedy C.
        • Vanhoucke V.
        • Ioffe S.
        • et al.
        Rethinking the inception architecture for computer vision. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.
        IEEE, Las Vegas (NV)2016: 2818-2826
      21. Lin T.-Y, Goyal P, Girshick R, et al. Focal loss for dense object detection. Proceedings of the IEEE International Conference on Computer Vision. Venice (Italy): IEEE; 2017. p. 2980–8.

      22. Sandler M, Howard A, Zhu M, et al. Mobilenetv2: Inverted residuals and linear bottlenecks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City (UT): IEEE; 2018. p. 4510–20.

      23. He K, Zhang X, Ren S, et al. Identity mappings in deep residual networks. European Conference on Computer Vision. Amsterdam (The Netherlands): Springer; 2016. p. 630–45.

      24. Ronneberger O, Fischer P, Brox T. U-Net: Convolutional networks for biomedical image segmentation. International Conference on Medical image computing and computer-assisted intervention. Munich (Germany): Springer; 2015. p. 234–41.

      25. Hashemi SR, Prabhu SP, Warfield SK, et al. Exclusive independent probability estimation using deep 3d fully convolutional densenets for isointense infant brain MRI segmentation, arXiv preprint arXiv:1809.08168.

        • Sun J.
        • Yan S.
        • Song C.
        • et al.
        Dual-functional neural network for bilateral hippocampi segmentation and diagnosis of alzheimer’s disease.
        Int J Comput Assist Radiol Surg. 2020; 15: 445-455
      26. Milletari F, Navab N, Ahmadi S.-A. V-net: Fully convolutional neural networks for volumetric medical image segmentation. 2016 Fourth International Conference on 3D Vision (3DV), IEEE, Stanford University, California, October 25, 2016. p. 565–71.

        • Yin S.
        • Peng Q.
        • Li H.
        • et al.
        Automatic kidney segmentation in ultrasound images using subsequent boundary distance regression and pixelwise classification networks.
        Med Image Anal. 2020; 60: 101602
      27. Simonyan K, Zisserman A. Very deep convolutional networks for largescale image recognition, arXiv preprint arXiv: 1409.1556.

      28. Chen L, Papandreou G, Kokkinos I, et al. Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs, arXiv preprint. arXiv:1606.00915.

      29. Chen L, Papandreou G, Kokkinos I, et al. Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs, arXiv preprint arXiv:1606.00915.

        • de Vos B.D.
        • Berendsen F.F.
        • Viergever M.A.
        • et al.
        A deep learning framework for unsupervised affine and deformable image registration.
        Med Image Anal. 2019; 52: 128-143
      30. Balakrishnan G, Zhao A, Sabuncu MR, et al. Voxelmorph: A learning framework for deformable medical image registration, arXiv preprint arXiv:1809.05231.

      31. Shen Z, Han X, Xu Z, et al. Networks for joint affine and nonparametric image registration, arXiv preprint arXiv:1903.08811.

      32. Eppenhof KAJ, Lafarge MW, Pluim JPW. Progressively growing convolutional networks for end-to-end deformable image registration. In: Angelini ED, Landman BA, editors. Medical imaging 2019: image processing, ociety of Photo-Optical Instrumentation Engineers (SPIE). Bellingham (WA): SPIE Press; 2019. p. 338–44.

      33. Yang H, Sun J, Carass A, et al. Unpaired brain MR-to-CT synthesis using a structure-constrained CycleGAN, arXiv preprint arXiv:1809.04536.

      34. Zhu J.-Y, Park T, Isola P, et al. Unpaired image-to-image translation using cycle-consistent adversarial networks. Proceedings of the IEEE International Conference on Computer Vision. Venice (Italy): IEEE; 2017. p. 2223–2232.

        • Rubin J.
        • Abulnaga S.M.
        CT-To-MR conditional generative adversarial networks for ischemic stroke lesion segmentation.
        IEEE Int Conf Healthc Inform. 2019; : 1-7
      35. Bass C, Dai T, Billot B, et al. Image synthesis with a convolutional capsule generative adversarial network. Proceedings of The 2nd International Conference on Medical Imaging with Deep Learning, Vol. 102 of Proceedings of Machine Learning Research. London (UK): PMLR; 2019. p. 39–62.

      36. LaLonde R, Bagci U, Capsules for object segmentation, arXiv preprint arXiv:1804.04241.

      37. Sabour S, Frosst N, Hinton GE, Dynamic routing between capsules, arXiv preprint arXiv:1710.09829.

      38. Sriram A, Zbontar J, Murrell T, et al, GrappaNet: Combining parallel imaging with deep learning for multi-coil MRI reconstruction, arXiv preprint arXiv:1910.12325.

        • Akagi M.
        • Nakamura Y.
        • Higaki T.
        • et al.
        Deep learning reconstruction improves image quality of abdominal ultra-high-resolution CT.
        Eur Radiol. 2019; 29: 6163-6171
        • Gondara L.
        Medical image denoising using convolutional denoising autoencoders. 2016 IEEE 16th International Conference on Data Mining Workshops (ICDMW).
        IEEE, Barcelona (Spain)2016: 241-246
        • Jifara W.
        • Jiang F.
        • Rho S.
        • et al.
        Medical image denoising using convolutional neural network: a residual learning approach.
        J Supercomput. 2019; 75: 704-718