No Access

Dunlap Institute for Astronomy and Astrophysics, University of Toronto, Toronto, Ontario, M5S 3H4, Canada

David A. Dunlap Department of Astronomy and Astrophysics, University of Toronto, Toronto, Ontario, M5S 3H4, Canada

Central Development Laboratory, National Radio Astronomy Observatory, Virginia, 22903, USA

E-mail Address: [email protected]

Search for more papers by this author

Andre Renard

http://orcid.org/0000-0003-3463-7918

Dunlap Institute for Astronomy and Astrophysics, University of Toronto, Toronto, Ontario, M5S 3H4, Canada

Search for more papers by this author

Keith Vanderlinde

http://orcid.org/0000-0003-4535-9378

Dunlap Institute for Astronomy and Astrophysics, University of Toronto, Toronto, Ontario, M5S 3H4, Canada

David A. Dunlap Department of Astronomy and Astrophysics, University of Toronto, Toronto, Ontario, M5S 3H4, Canada

Search for more papers by this author

Philippe Berger

http://orcid.org/0000-0003-3322-3510

Canadian Institute for Theoretical Astrophysics, University of Toronto, Toronto, Ontario, M5S 3H4, Canada

Jet Propulsion Laboratory, California Institute of Technology, Pasadena, California, 91109, USA

Search for more papers by this author

Kiyoshi Masui

http://orcid.org/0000-0002-4279-6946

MIT Kavli Institute for Astrophysics and Space Research, Massachusetts Institute of Technology, Cambridge, Massachusetts, 02109, USA

Search for more papers by this author

Ian Tretyakov

http://orcid.org/0000-0002-7436-2325

Dunlap Institute for Astronomy and Astrophysics, University of Toronto, Toronto, Ontario, M5S 3H4, Canada

Department of Physics, University of Toronto, Toronto, Ontario, M5S 3H4, Canada

Search for more papers by this author

and

The CHIME Collaboration

Search for more papers by this author

https://doi.org/10.1142/S2251171720500142Cited by:4

View article

Abstract

We present an overview of the Graphics Processing Unit (GPU)-based spatial processing system created for the Canadian Hydrogen Intensity Mapping Experiment (CHIME). The design employs AMD S9300x2 GPUs and readily available commercial hardware in its processing nodes to provide a cost- and power-efficient processing substrate. These nodes are supported by a liquid-cooling system which allows continuous operation with modest power consumption and in all but the most adverse conditions. Capable of continuously correlating 2048 receiver-polarizations across 400MHz of bandwidth, the CHIME X-engine constitutes the most powerful radio correlator currently in existence. It receives 6.6Tb/s of channelized data from CHIME’s FPGA-based F-engine, and the primary correlation task requires $8.39 \times 1 0^{14}$ complex multiply-and-accumulate operations per second. The same system also provides formed-beam data products to commensal FRB and Pulsar experiments; it constitutes a general spatial-processing system of unprecedented scale and capability, with correspondingly great challenges in computation, data transport, heat dissipation, and interference shielding.

Keywords:

References

Ali, Z. S., Parsons, A. R., Zheng, H. et al. [2015] ApJ 809, 61. https://doi.org/10.1088/0004-637X/809/1/61 Crossref, ADS, Google Scholar
Bandura, K., Addison, G. E., Amiri, M. et al. [2014] “Canadian Hydrogen Intensity Mapping Experiment (CHIME) pathfinder,” Proc. SPIE 9145, 914522. https://doi.org/10.1117/12.2054950 Crossref, Google Scholar
Bandura, K., Bender, A. N., Cliche, J. F. et al. [2016a] J. Astron. Instrum. 5, 1641005. https://doi.org/10.1142/S225117171 6410051 Link, ADS, Google Scholar
Bandura, K., Cliche, J. F., Dobbs, M. A. et al. [2016b] J. Astron. Instrum. 5, 1641004. https://doi.org/10.1142/S22511717 1641004X Link, Google Scholar
Booth, R. S. & Jonas, J. L. [2012] Afr. Skies 16, 101. ADS, Google Scholar
Brewer, R. [2001] Shielding: A Look at Waveguide Penetrations, Evaluation Engineering, https://www.evaluationengineering.com/home/article/13001785/shielding-a-look-at-waveguide-penetrations. Google Scholar
CHIME Scientific Collaboration [2020] In prep. Google Scholar
CHIME/FRB Collaboration ( Amiri, M., Bandura, K., Berger, P. et al.) [2018] ApJ 863, 48. https://doi.org/10.3847/1538-4357/aad188 Crossref, ADS, Google Scholar
CHIME/FRB Collaboration ( Amiri, M., Bandura, K., Bhardwaj, M. et al.) [2019a] Nature 566, 235. https://doi.org/10.1038/s41586-018-0864-x Crossref, ADS, Google Scholar
CHIME/FRB Collaboration ( Amiri, M., Bandura, K., Bhardwaj, M. et al.) [2019b] Nature 566, 230. https://doi.org/10.1038/s41586-018-0867-7 Crossref, ADS, Google Scholar
CHIME/FRB Collaboration (Andersen, B. C., Bandura, K. M., Bhardwaj, M. et al.) [2020a] arXiv:2005.10324. Google Scholar
CHIME/FRB Collaboration ( Amiri, M., Andersen, B. C., Bandura, K. M. et al.) [2020b] Nature 582, 351. https://doi.org/10.1038/s41586-020-2398-2 Crossref, ADS, Google Scholar
CHIME/Pulsar Collaboration (Amiri, M., Bandura, K. M., Boyle, P. J. et al.) [2020] arXiv:2008.05681. Google Scholar
Cisco [2018] “Cisco visual networking index: Forecast and trends, 2017–2022,” Tech. Rep. C11-741490-00, Cisco Systems Inc. Google Scholar
Clark, M. A., La Plante, P. C. & Greenhill, L. J. [2011] arXiv:1107.4264. Google Scholar
DeBoer, D. R., Parsons, A. R., Aguirre, J. E. et al. [2017] PASP 129, 045001. https://doi.org/10.1088/1538-3873/129/974/045001 Crossref, Google Scholar
Deng, M. & Campbell-Wilson, D. [2014] “The cloverleaf antenna: A compact wide-bandwidth dual-polarization feed for CHIME,” in 2014 16th Int. Symp. Antenna Technology and Applied Electromagnetics (ANTEM) (Victoria, BC), pp. 1–2. https://doi.org/10.1109/ANTEM.2014.6887670 Crossref, Google Scholar
Denman, N. T. [2019] Digital Signal Processing for the Canadian Hydrogen Intensity Mapping Experiment, Ph.D. Thesis, University of Toronto. Google Scholar
Dewdney, P., Turner, W., Braun, R. et al. [2015] “SKA1 system baseline v2 description,” Tech. Rep. SKA-TEL-SKO-0000308, SKA Organization. Google Scholar
Intel Corporation [2018] Intel 64 and IA-32 Architectures Software Developer’s Manual (Intel Corporation). Google Scholar
Klages, P., Bandura, K., Denman, N. et al. [2015] “GPU kernels for high-speed 4-bit astrophysical data processing,” in Proc. IEEE 26th Int. Conf. Application-Specific Systems, Architectures and Processors (Toronto, ON), pp. 164–165. Crossref, Google Scholar
Kocz, J., Greenhill, L. J., Barsdell, B. R. et al. [2015] J. Astron. Instrum. 4, 1550003. https://doi.org/10.1142/S2251171715 500038 Link, Google Scholar
Limoncelli, T. A., Hogan, C. J. & Chalup, S. R. [2007] The Practice of System and Network Administration (Addison-Wesley Professional). Google Scholar
Loeb, A. & Wyithe, J. S. B. [2008] Phys. Rev. Lett. 100, 161301. https://doi.org/10.1103/PhysRevLett.100.161301 Crossref, ADS, Google Scholar
NVIDIA Corporation [2019] “Parallel thread execution ISA version 6.5,” https://docs.nvidia.com/cuda/parallel-thread-execution/. Google Scholar
Ng, C., Vanderlinde, K., Paradise, A. et al., [2017] arXiv:1702.04728. Google Scholar
Ng, C. [2017] arXiv:1711.02104. Google Scholar
Ord, S. M., Crosse, B., Emrich, D. et al. [2015] Publ. Astron. Soc. Aust. 32, e006. https://doi.org/10.1017/pasa.2015.5 Crossref, ADS, Google Scholar
Perley, R., Napier, P., Jackson, J. et al. [2009] IEEE Proc. 97, 1448. https://doi.org/10.1109/JPROC.2009.2015470 Crossref, ADS, Google Scholar
Pritchard, J. R. & Loeb, A. [2012] Rep. Prog. Phys. 75, 086901. https://doi.org/10.1088/0034-4885/75/8/086901 Crossref, Google Scholar
Renard, A. et al. [2020] In prep. Google Scholar
Romein, J. & Veenboer, B. [2019] “Extreme signal-processing performance using tensor cores and astronomical imaging on GPUs,” GTC Silicon Valley Session S9306. Google Scholar
Romney, J. D. [1999] “Cross correlators,” in Synthesis Imaging in Radio Astronomy II, eds. Taylor, G. B.Carilli, C. L.Perley, R. A. (Astronomical Society of the Pacific), pp. 57–78. Google Scholar
Sawyer, R. [2004] “Calculating total power requirements for data centers,” Tech. Rep. 3, American Power Conversion Corporation. Google Scholar
Selina, R. J., Murphy, E. J., McKinnon, M. et al. [2018] “The Next-Generation Very Large Array: A technical overview,” Proc. SPIE 10700, 107001O. https://doi.org/10.1117/12.2312089 Crossref, Google Scholar
Tegmark, M. & Zaldarriaga, M. [2009] Phys. Rev. D 79, 083530. https://doi.org/10.1103/PhysRevD.79.083530 Crossref, ADS, Google Scholar
Tegmark, M. & Zaldarriaga, M. [2010] Phys. Rev. D 82, 103501. https://doi.org/10.1103/PhysRevD.82.103501 Crossref, ADS, Google Scholar
Thompson, A. R., Moran, J. M. & Swenson, G. W., Jr. [2001] Interferometry and Synthesis in Radio Astronomy (Wiley). Crossref, Google Scholar
Warmels, R., Biggs, A., Cortes, P. A. et al. [2018] “Alma technical handbook,” Tech. Rep. 6.3, ALMA Observatory. Google Scholar
Wayth, R. B., Tingay, S. J., Trott, C. M. et al. [2018] arXiv:1809.06466. Google Scholar
Weinberg, D. H., Mortonson, M. J., Eisenstein, D. J. et al. [2013] Phys. Rep. 530, 87. https://doi.org/10.1016/j.physrep.2013. 05.001 Crossref, ADS, Google Scholar
Wilson, T. L., Rohlfs, K. & Hüttemeister, S. [2009] Tools of Radio Astronomy (Springer-Verlag). https://doi.org/10.1007/978-3-540- 85122-6 Google Scholar

Published: 13 October 2020

Vol. 09, No. 03

Metrics

Downloaded 25 times

History

Received 19 May 2020

Revised 8 September 2020

Accepted 9 September 2020

Keywords

PDF download

System Upgrade on Mon, Jun 21st, 2021 at 1am (EDT)

A GPU Spatial Processing System for CHIME

Abstract

References

System Upgrade on Mon, Jun 21st, 2021 at 1am (EDT)

A GPU Spatial Processing System for CHIME

Abstract

References

Recommended