Arxiv Package Analysis

Author

Santiago Rodriguez

Published on Nov 05, 2024

About

Howdy

As often happens with curious minds, I grew curious about something. Recently, I was browsing the statistics section of Arxiv for papers on functional data analysis when a thought came to me - I wonder what [coding] languages are most prevalent on Arxiv?

For those unfamiliar, from the Arxiv website:

Arxiv is a free distribution service and an open-access archive for nearly 2.4 million scholarly articles in the fields of physics, mathematics, computer science, quantitative biology, quantitative finance, statistics, electrical engineering and systems science, and economics. Materials on this site are not peer-reviewed by arXiv.

Since the original thought is a bit broad, an approximiation will have to suffice. Instead of analyzing the use of all languages across all sections for all purposes the scope of the analysis will focus on the mention of packages in the statistics section. The coding languages to be analyzed are Julia, Python, and R.

Overall

  • R:
    • search terms: “r package”
    • 1,789 results
    • first mentioned in 2005
  • Python:
    • search terms: “python package” OR “python module”
    • 241 results
    • first mentioned in 2011
  • Julia:
    • search terms: “julia package”
    • 19 results
    • first mentioned in 2018

Overall, R packages have been more frequently discussed that Python or Julia packages in the statistics section of Arxiv.

Results by Month

R

For years mentions of “r package” on Arxiv grew steadily. In the past few years though, the number of mentions of have trended downward.

Python

Meanwhile mentions of “python package” OR “python module” have slowly trended upward. I wonder if the rate of Python mentions will increase similar to those of the R mentions?

Julia

With only 19 Julia results there are too few mentions to make an inference. However, I am a big fan of Julia so I hope to see more posts on Arxiv about Julia packages.

Conclusion and Next Steps

I’m not surprised that R mentions were most prevalent as R and statistics go hand-in-hand. However, unsurprisingly, Python mentions have increased since 2011. I’d definitely like to see more Julia mentions.

Well, my curiosity has been satisifed so I don’t plan on pursuing this further. However, I do have some thoughts about what I could do with this information.

  • time series forecasting
    • great, so we know what mentions have been but where will mentions be next month, year, etc.?
  • functional data analysis
    • plot smoothed curves as the raw data is quite jagged
    • analyze the rate of change of the different coding languages (i.e, derivatives)
  • visualizations
    • practice displaying the information presented here in different formats

Appendix

Thank you to arXiv for use of its open access interoperability.