Category Archives: special sessions

Report on the Astronomy Software Publishing Special Session at AAS231

On Thursday, January 11, the Astrophysics Source Code Library (ASCL) and Astronomical Data Group at the Flatiron Institute organized a Special Session at the 231st AAS meeting in National Harbor, MD on Astronomy Software Publishing: Community Roles and Services, the sixth in a series of software-focused sessions that the ASCL, sometimes with others, has organized at AAS meetings.

"Really glad to see software article publication and citation getting attention at the #AAS231 meeting. Great articles like Daniel Foreman-Mackey's "emcee: The MCMC Hammer in PASP is a perfect example of a highly-referenced software article."Peter Teuben from the University of Maryland and chair of the ASCL’s Advisory Committee) opened the session with a few words about the use of software in research articles. He outlined the layout of the session. A talk by Matteo Cantiello set the scene on how we have reached the point where we are now. Four presentations by representatives from different journals presented their policies on software publication followed Cantiello’s talk, and they were followed by presentations by representatives of others with roles in publishing software: the software author, the data editor, the ADS and the ASCL. The floor was then opened for discussion and Q&A. Teuben moderated the discussion, and at the end of it, turned the podium over to Robert Nemiroff from Michigan Technological University, and a founder of the ASCL, for a summary and closing remarks.

Some of the main points from each presentation are summarized below; the titles of each are links to the slides used by the presenters.

  • The Evolution of Software Publication in Astronomy, Matteo Cantiello (Flatiron Institute)
    Cantiello states that the complexity of astrophysics requires computationally intensive models, making astronomy a digital science, and that astronomers have a rich computational environment available, allowing them to easily version, share, and deploy astronomy software. Reproducibility paradoxDespite this, software is often not shared, resulting in a reproducibility paradox: astronomers use computation to provide precise, accurate results, but research has become less transparent with the increase in the use of computational methods. Adding external links to papers to link to software is not a reliable solution to software sharing because of link rot. Formats have changed very little in the last 400 years; despite progress both technologically and socially, the format of papers is still largely the same. He stated that astronomy now has an opportunity to rethink scientific papers as research repositories, with executable objects containing narrative, figures, data, and code.
  • Software papers and citation in the AAS Journals, Chris Lintott (AAS Journals)
    The AAS journals policy on software until recently was set in 1964, which stated that the “need for communication between astronomers interested in computation is already supplied by associations of users of automated computing machines.” The AAS journals changed their policies at the beginning of 2016, and recognized that if novel code is important to published research then it is likely appropriate to describe it in such a paper. Papers can be short, descriptive, and need not include research resultsAAS journals are interested in disclosing software in a form that is currently recognized: the research article, so now allow short papers on code that can be short, descriptive, and do not need to include scientific results. AAS formally recommends open source licensing but does not require it. AAS journals ask people to cite the software paper, as this is the currency the field cares about and also ask people to cite the code. In addition, they request people use the \software{} tag to create a software section in a paper; this is similar to the \facilities{} tag already in use. AAS Publishing continues to think about how to improve, and are introducing the concept of ‘living’ papers, which can be updated with new sections and expanded author lists, so software authors don’t need to publish a new paper to give credit to software authors who have contributed to a new version of the software. Lintott encouraged those interested in living papers to contact him.
  • Software policies and guidelines at Nature, Leslie J. Sage (Nature)
    First, Sage explained the context in which Nature‘s policy is created: Nature is driven by biologists, who live in a very different world from astronomers. Unlike astronomers, biologists live in Windows world. Right now, two journals, Nature Methods and Nature Biotech, require code to be made available, and there are ongoing discussions about whether Nature should do this for other journals. There are formidable problems because of the issue of very specialized code, for example, code that is optimized to compile on a particular Beowulf cluster that may not compile anywhere else. There will be a call for public comment, and Sage hopes astronomers will provide input that is useful for astronomers within that context. Sage raised a number of points that warrant public discussion, such as a preference voiced by some to see detailed descriptions of the algorithms used rather than having the scripts published. Another point to consider for input is that though a lot of software has been made publicly available, all software is written with certain constraints and boundary conditions; some people not aware of these constraints and conditions may drive the software beyond those limits; this raises the question as to whether the results are physically meaningful.
  • SpringerNature data and software policies for astrophysics journals, Ramon Khanna (Springer)
    Springer is encouraging authors to take care of transparency and reproducibility of their results presented in articles, allowing them to append relevant information on source code or the full code in an appendix of the paper; authors can also append the full code, or use other methods to provide this information, such as alternative repositories (e.g., CDS, ASCL, Figshare), and making this information available. They would like the full data and code available. Khanna acknowledged some challenges, including that authors are often not willing to share their software and/or data, editors are often not willing or at least not determined enough to execute policy, and citation standards are unclear. One of the questions is how to execute this policy in the face of unwillingness from authors, editors, reviewers, etc.Questions arise as to how software can be peer reviewed; this would require standards for documentation,  presenting how the results were obtained, making data and software available, and for reviewing the source code itself. How can referees handle this effort? Khanna pointed out that in a field as advanced as astronomy is, and already having some standards and domain resources such as archives, it’s not so much the publisher that should drive new standards, but the community itself.
  • Journal of Open Source Software (JOSS): Design and first-year review, Arfon M. Smith (STScI/JOSS)
    Smith stated that he created JOSS accidentally, from frustration about the overhead of publishing papers about software, and acknowledged that software papers are a hack of the current system to provide a citable, creditable research object for software.  JOSS ( seeks to improve the quality of software; its peer review process is almost entirely about the software that’s submitted, and includes making sure the documentation is sufficiently fleshed out, that the package includes automated tests, and that the software has an open source license so can be reused. Smith said it should take about an hour to write a one-page paper for JOSS for those with a well set up repository for their code. The reviews are public on Github and accepted submissions appear on the JOSS site, which has published 200 papers online.
  • Lessons Learned through the Development and Publication of AstroImageJ, Karen Collins (Center for Astrophysics)
    Collins discussed her experience with publishing her software AstroImageJ, a data reduction and image display interface with analysis capabilities specialized for time series differential photometry. She developed the code over several years to support her research. She initially had no intention of releasing the code to the public, but her collaborators saw her plots and graphs and asked to use the software, which was posted to the university’s website to give team members access to it. AstroImageJ lessons learnedShe found her fellow KELT-FUN team members were an excellent focus group; they provided great feedback on the software before it was published, enabling her to add useful features to the software. Results using AstroImageJ started appearing in journals; she registered the software with the ASCL to give it a citable reference, and as usage (and support tasks) grew, she and others working on the code decided to submit a paper to the Astronomical Journal (AJ) to provide good exposure to the potential userbase for the software. This resulted in about 4K downloads of the software in the first year, and the paper is listed 4th on AJ’s most read list. Among the lessons learned in publishing AstroImageJ are to specify how your code is licensed and how it should be cited, make the source code easily accessible, and provide easy way to install and update the software.
  • The roles of the AAS Journals’ Data Editors, August Muench (AAS Journals)
    Muench covered the data editors’ workflow for all submitted manuscripts. A quick review of 60%-90% of all submitted manuscripts is performed, with scripts run on the manuscripts to identify references to code by looking for such things as Github repositories to see whether  their citations need to be reviewed. The editors make notes on the software, data, and figures for review by a scientific editor or the author with recommendations for improving citations for these research artifacts. A subset of accepted articles, 15-20%, undergo a more rigorous post-acceptance data review; this includes a review of tabular data, figures, and interactive elements in addition to software. If necessary, the data editors request that authors acquire DOIs or get preferred citati"People recognize software via plots (and other fingerprints). Make sure you cite the code. I still recognize plots made with PAW."ons for the software used in the research. Muench mentioned that he uses ten keywords in his scripts to identify software, and ends up with a surprising number of articles that do not mention code at all. He stated that part of a data editor’s role to improve software and data citation is educating authors.
  • The role of the ADS in software discovery and citation, Alberto Accomazzi (NASA Astrophysics Data System)
    Accomazzi described what ADS does to promote software discovery and citation, but first he shared ADS’s traditional core responsibilities: to discover content, typically science papers, related to astronomy. Some years ago, the capability to track citations was introduced. As the expectations of community have evolved, so have ADS’s policies, moving from ingesting records about scientific papers to records about scholarly works, including data catalogs, observing proposals, and other artifacts such as software. They have also evolved from tracking citations to articles to citations to tracking citations to scholarly content. How ADS awards citationsADS has an interest in enabling linking so users can easily and uniquely identify the software that was used. Accomazzi covered how ADS ingestion works; for content to be considered for inclusion in ADS, it must be scholarly, related to astronomy, and published formally — not just on a website, but following an explicit editorial process. He also discussed how citations are tracked and what ADS needs to count a citation, going through several examples of what does and does not work for citation. The bottom line for software is to cite it by using a formal citation and a unique identifier; a URL to a website or a DOI in a footnote are not captured as citations. ASCL, JOSS, and Zenodo are ways software can get a persistent identifier to use in a formal citation, and these citations can be tracked by ADS. Accomazzi also discussed how software may have several records in ADS, and that in the future, these records will be crosslinked, as will different versions of a software package so that eventually, ADS can provide cumulative metrics for all different versions of that software product, and like all citation data, this information will become publicly available through an API.
  • The Astrophysics Source Code Library: Supporting software publication and citation, Alice Allen (ASCL/UMD)
    Allen gave a brief overview of what the ASCL is, and stated that though entries in this citable online registry usually point to a software package’s download it, the ASCL can and does serve as a repository for those authors who want to deposit an archive file for their code. The ASCL assigns a DOI to software that it stores. She covered the three main reasons the ASCL exists: to make research more transparent, to improve communication about research computations, and to disseminate software of utility to others. Allen acknowledged that though there is software that might be useful to astronomy, the ASCL focuses on that which has been used in refereed research or submitted for refereeing, this to support the research record. ASCL editors take an active approach in looking for software in research papers and registering them; authors are encouraged to submit their own software, too, and submission by author increased 23% in 2017 over 2016. The ASCL supports software publication and citation in a number of ways, including providing a citation avenue for software and listing preferred citation information in ASCL entries. The ASCL has been online since 1999; it supports the Force11 software citation principles and was a party to  developing them. It was also party to a Dagstuhl Manifesto, another cross-disciplinary effort  that focused on steps members of a research community can take on their own. Among these steps is citing software properly — in a trackable way — and when reviewing a paper, ensuring that it cites the software used in the research.

After the presentations, Teuben commented that he thought journals could do a better job in instructing referees about software, to identify when code is involved in research and insist on citations to it. He hoped the discussion would touch on this, and then opened the floor to all. People interested in software sustainability might want to follow @si2urssi ( which is working to plan a US research software systainability institute - a first workshop will likely be in AprilDiscussion was lively and may be covered in more depth in a future post, but some of the major points were:

  • There’s still fear about releasing software, still resistance to doing so
  • Science is all about reproducibility; it’s not science if it’s not reproducible
  • Who should push for greater openness is an open question, with some wanting journals to do this, and others feeling it’s up to the astronomy community — us! — to enforce the standards we want
  • Astronomers are often not trained in software engineering techniques; greater education in this area would be helpful

"If software developers were well funded, ie would be easier to get people to share their code."

Teuben brought the discussion to an end and turned the floor over to Robert Nemiroff (Michigan Technological University), who briefly summarized the presentations and discussion and closed the session.

My thanks to David W. Hogg and Peter Teuben for work on developing the session, to Peter for his excellent moderating, to Robert for closing the session, and for PW Ryan for serving as scribe. My thanks to Matteo, Chris, Leslie, Ramon, Arfon, Karen, Gus, and Alberto for their excellent presentations and participation, to the Astronomical Data Group at the Flatiron Institute for partnering with the ASCL, and to the Heidelberg Institute for Theoretical Studies, the University of Maryland College Park, and Michigan Technological University for supporting the ASCL.

Slides from Astronomy Software Publishing: Community Roles and Services

Special Session: Astronomy Software Publishing: Community Roles and Services
10:00 am – 11:30 am
National Harbor 2

The Astrophysics Source Code Library (ASCL) and Astronomical Data Group at the Flatiron Institute organized a Special Session at the 231st AAS meeting in National Harbor, MD on Astronomy Software Publishing: Community Roles and Services. Click on a talk’s title to download its slides.

Matteo Cantiello (Flatiron Institute), The Evolution of Software Publication in Astronomy
Chris Lintott (AAS Journals), Software papers and citation in the AAS Journals
Leslie J. Sage (Nature), Software policies and guidelines at Nature
Ramon Khanna (Springer), SpringerNature data and software policies for astrophysics journals
Arfon M. Smith (STScI/JOSS), Journal of Open Source Software (JOSS): Design and first-year review
Karen Collins (Center for Astrophysics), Lessons Learned through the Development and Publication of AstroImageJ
August Muench (AAS Journals), The roles of the AAS Journals’ Data Editors
Alberto Accomazzi (NASA Astrophysics Data System), The role of the ADS in software discovery and citation
Alice Allen (ASCL/UMD), The Astrophysics Source Code Library: Supporting software publication and citation

ASCL at AAS 231

It’s AAS meeting time, and that means a busy busy week! There are some of the ASCL’s activities at this meeting; we hope to see you at our posters and in the Special Session!

Poster 150.10: The Astrophysics Source Code Library by the numbers
Tuesday, January 09, Prince Georges Exhibit Hall

Abstract: The Astrophysics Source Code Library (ASCL, was founded in 1999 by Robert Nemiroff and John Wallin. ASCL editors seek both new and old peer-reviewed papers that describe methods or experiments that involve the development or use of source code, and add entries for the found codes to the library. Software authors can submit their codes to the ASCL as well. This ensures a comprehensive listing covering a significant number of the astrophysics source codes used in peer-reviewed studies. The ASCL is indexed by both NASA’s Astrophysics Data System (ADS) and Web of Science, making software used in research more discoverable. This presentation covers the growth in the ASCL’s number of entries, the number of citations to its entries, and in which journals those citations appear. It also discusses what changes have been made to the ASCL recently, and what its plans are for the future.

Poster 150.28: Schroedinger’s code: Source code availability and transparency in astrophysics
Tuesday, January 09, Prince Georges Exhibit Hall

Abstract: Astronomers use software for their research, but how many of the codes they use are available as source code? We examined a sample of 166 papers from 2015 for clearly identified software use, then searched for source code for the software packages mentioned in these research papers. We categorized the software to indicate whether source code is available for download and whether there are restrictions to accessing it, and if source code was not available, whether some other form of the software, such as a binary, was. Over 40% of the source code for the software used in our sample was not available for download.

As URLs have often been used as proxy citations for software, we also extracted URLs from one journal’s 2015 research articles, removed those from certain long-term, reliable domains, and tested the remainder to determine what percentage of these URLs were still accessible in September and October, 2017.

Special Session: Astronomy Software Publishing: Community Roles and Services
Thursday, January 11, 10:00 AM – 11:30 AM; National Harbor 2
Organizers: Astrophysics Source Code Library (ASCL)/Astronomical Data Group at the Flatiron Institute

The importance of software to astronomy research is well-established, and excellent arguments to reveal these computational methods to support the research record have been advanced and much discussed in recent years. But what avenues are open to software authors to publish their codes, and what roles and services exist in the community to support their efforts? This session, organized by the Astrophysics Source Code Library (ASCL) and Astronomical Data Group at the Flatiron Institute, answers that question. It builds on previous AAS special sessions and brings together a panel of experts to present on the various avenues for publishing codes and the pros and cons of these avenues, the roles of authors, data editors, and publication indexers in software publication, the benefits of publication to authors and the discipline, and efforts of related community projects to improve aspects of software publication. After the presentations, the floor will be open for discussion and questions.

The topics and panelists are:

Introductory remarks, Peter Teuben (University of Maryland)
The evolution of software publication in astronomy, Matteo Cantiello (Flatiron Institute, Center for Computational Astrophysics/Princeton University)
Software papers and citation in the AAS journals, Chris Lintott (AAS Journals/University of Oxford)
Software policies and guidelines at Nature, Leslie J. Sage (Physical Sciences, Nature)
SpringerNature data and software policies for astrophysics journals, Ramon Khanna (Astronomy, Springer)
Journal of Open Source Software (JOSS): design and first-year review, Arfon M. Smith (Space Telescope Science Institute)
Lessons learned through the development and publication of AstroImageJ, Karen Collins (Harvard-Smithsonian Center for Astrophysics)
The roles of the AAS Journals’ Data Editors, August Muench (Journals Division, AAS)
The role of the ADS in software discovery and citation, Alberto Accomazzi (Smithsonian Astrophysical Observatory)
The Astrophysics Source Code Library: Supporting software publication and citation, Alice Allen (ASCL/University of Maryland)
Open discussion and Q&A, Moderated by Peter Teuben (University of Maryland)
Summary and closing remarks, Robert J. Nemiroff (Michigan Technological University)

Astronomy Software Publishing: Community Roles and Services Special Session at AAS 231

Special Session: Astronomy Software Publishing: Community Roles and Services
10:00 am – 11:30 am
National Harbor 2

The Astrophysics Source Code Library (ASCL) and Astronomical Data Group at the Flatiron Institute have organized a Special Session at January’s AAS meeting. The session, Astronomy Software Publishing: Community Roles and Services, will be moderated by Peter Teuben (University of Maryland).

The importance of software to astronomy research is well-established, and excellent arguments to reveal these computational methods to support the research record have been advanced and much discussed in recent years. But what avenues are open to software authors to publish their codes, and what roles and services exist in the community to support their efforts? This session answers these questions. It builds on previous AAS special sessions and brings together a panel of experts to present various avenues for publishing codes and the pros and cons of these avenues, the roles of authors, data editors, and publication indexers in software publication, the benefits of publication to authors and the discipline, and efforts of related community projects to improve aspects of software publication. After the presentations, the floor will be open for discussion and questions. The presenters and topics covered are:

Matteo Cantiello (Flatiron Institute), The Evolution of Software Publication in Astronomy
Chris Lintott (AAS Journals), Software papers and citation in the AAS Journals
Leslie J. Sage (Nature), Software policies and guidelines at Nature
Ramon Khanna (Springer), SpringerNature data and software policies for astrophysics journals
Arfon M. Smith (STScI/JOSS), Journal of Open Source Software (JOSS): Design and first-year review
Karen Collins (Center for Astrophysics), Lessons Learned through the Development and Publication of AstroImageJ
August Muench (AAS Journals), The roles of the AAS Journals’ Data Editors
Alberto Accomazzi (NASA Astrophysics Data System), The role of the ADS in software discovery and citation
Alice Allen (ASCL/UMD), The Astrophysics Source Code Library: Supporting software publication and citation

And you, astro software authors and users, are as always important participants in the discussion. Please come with your questions, observations, and comments; I hope to see you there!

Developments and Practices in Astronomy Research Software at EWASS 2017

The annual meeting of the European Astronomical Society (EAS), the European Week of Astronomy and Space Science (EWASS), was held June 26-30 in beautiful Prague. As mentioned in a previous blog post, Abigail Stevens (U Amsterdam), Amruta Jaodand (ASTRON), Matteo Bachetti (INAF-Osservatorio Astronomico di Cagliari), Rein Warmels (ESO), and I organized a Special Session titled Developments and Practices in Astronomy Research Software. Special Sessions at EWASS can be from one to three 90-minute blocks; we organized three blocks, each with its own focus and all scheduled on Wednesday, June 28, which were:

Best practices for code development and management
Perspectives in research software
Astronomy software packages review

The first two sessions opened with review talks, then had a series of slightly shorter presentations, each with a particular focus. Each of these sessions concluded with a 30-minute period in which the floor was open to all to ask questions of the speakers and discuss the issues and information that had been shared during the 90 minutes. The third session offered talks on specific popular software packages and concluded with lightning talks on the software posters submitted to this Special Session. In addition to this series of talks, a Hack Together Day was organized collaboratively that offered more information on and assistance with installing and/or using many of the software packages presented.

Session titles and presenters for the software sessions that Abigail Stevens (U Amsterdam), Amruta Jaodand (ASTRON), Matteo Bachietti (INAF-Osservatorio Astronomico di Cagliari), Rein Warmels (ESO), and I organize are listed below, and if you want to read through the Storify of session tweets, compiled by Stevens as a Hack Together Day project, that’s here:

Best practices for code development and management
Moderator: Rein Warmels, ESO, Germany

This session opened with a talk by Simon Portegies Zwart (Leiden University, NL), author of AMUSE and other astro codes, on Reproducible science in scientific computing. He laid the groundwork for this block of talks by presenting the difficulties of reproducibility in simulation software, the best practices his group uses, and the philosophy behind AMUSE, which includes standardizing interfaces and automating as much as possible. His talk included simulations that made for an extremely large presentation file, so a partial set of his slides is provided.
(slides: PDF)

Simon Portegies Zwart presenting at EWASS

Simon Portegies Zwart

In the interest of time, I will not summarize the other talks in this block, but will say you should have been there! These were excellent presentations with many great practices and ideas shared. In the discussion, moderated by Warmels, someone asked about “short codes,” and as luck would have it, the next block had a talk on just that topic. There was disagreement on some voiced opinions, and many ideas shared that warrant greater discussion.

The other talks in this block were:

Software development best practices from Astropy
Thomas Robitaille, Freelance, UK (slides: PDF)

A Computer Science Perspective on the Astronomy Research Software Process
John Wenskovitch, Virginia Tech & Allegheny College, US (slides: PDF)

TARDIS: A radiative transfer code, an open source community, and an interdisciplinary collaboration
Wolfgang Kerzendorf, ESO, DE (slides: PDF)

Research software best practices: Transparency, credit, and citation
Alice Allen, ASCL, US (slides: PDF, PPTX)

Perspectives in research software
Moderator: Alice Allen, ASCL, USA
Kai Polsterer (Heidelberg Institute for Theoretical Studies, DE) set the stage for the second block in the Special Session with his presentation Reproducibility in Era of Data-Driven Science. He also highlighted the difficulties of reproducibility, among them that different computing environments can produce different results from the same code, and though he acknowledged the difficulties in doing so, advocates that publications, datasets (including raw and training data), codes, component and software configuration, and computing environments need to be shared for full reproducibility. Or as John Wenskovitch summarized in a tweet during this presentation, “Publish everything. EVERYTHING. Architecture, model, code, data, parameters, …” We are not there yet, but must share what we can when we can to increase reproducibility.
(slides: PDF)

Kai Polsterer, image by @K_Bonson

The other talks in this second of three blocks were:

Should short codes used for astronomy research be made public?
Robert Nemiroff, Michigan Technological University, US (slides: PDF, PPTX)

Giving credit where credit is due: the role of ADS in discovering and citing software in scholarly publications
Sergi Blanco-Cuaresma, Harvard-Smithsonian Center for Astrophysics, US (slides: PDF)

Fifteen years of WISE technology software development and operations
Gijs Verdoes Kleijn, University of Groningen, NL (slides: PDF)

CDS reference services supporting astronomy research
Mark Allen, CDS, Observatoire Astronomique De Strasbourg, FR (slides: PDF)

Astronomy software packages review
Moderator: Amruta Jaodand, ASTRON, NL

The third block of talks in this Special Session presented software packages useful for software research, from the well-established AstroPy to, in the short poster presentations, newly-developed software such as SPARTAN. The talks in this 90-minute block were:

The Astropy Project
Thomas Robitaille, Freelance, UK (slides: PDF)

Stingray and Dave: Spectral timing for all
Matteo Bachetti, INAF-Osservatorio Astronomico di Cagliari, IT (slides: PDF)

Living on the fringe: Making CASA ready for VLBI
Ilse van Bemmel, JIVE, NL

Interactive widgets for the Jupyter notebook
Maarten Breddels, Kapteyn Astronomical Institute, NL (Jupyter notebook, ipywidgets demo (PDF), poster)

Lightning poster talks

Again, you should have been there! One blog post cannot possibly convey everything shared in this Special Session, but the slides the presenters have shared captures some of the great goodness in these talks. My thanks to all the presenters, to my co-organizers, and to the attendees who made this session so interesting and excellent, to EWASS for accepting our proposals, and to HITS for providing the ASCL with funding that allowed us to participate.

ASCL at the 2017 European Week of Astronomy and Space Science (EWASS) meeting

The European Week of Astronomy and Space Science (EWASS) was held June 26-30 in Prague and attended by over 1,100 people, and the ASCL was there! This post is an overview of the ASCL’s participation in the event; a subsequent post (or two) will provide more detailed information and links to slide decks for sessions the ASCL was involved in organizing.

Program page for software talks

Program page for software talks
Image by Amruta Jaodand

This was my first time attending EWASS, which was initially brought to my attention by Keith Smith (Science). It was also my first time in Prague. My activities since the meeting have included submitting proposals (with others) for EWASS 2018, which will be in Liverpool, and pricing short-term apartment rentals in Prague; clearly, I liked both the meeting and the city very much! My thanks to Keith for cluing me in to this fine meeting.

ASCL Advisory Committee member Rein Warmels (ESO) and I partnered with Abigail Stevens (U Amsterdam), Amruta Jaodand (ASTRON), and Matteo Bachetti (INAF-Osservatorio Astronomico di Cagliari) on software-related sessions for EWASS 2017; our collaboration resulted in a day of talks on Wednesday called Developments and Practices in Astronomy Research Software and a Hack Together Day on Thursday, this latter coordinated by Stevens, Jaodand, and Bachetti.

The ASCL was well represented, with ASCL co-founder Robert Nemiroff (MTU) giving a talk on short codes and Warmels and I each moderating 90-minute sessions on software, both with a discussion period; I also gave a presentation on the ASCL and participated in the Hack Together Day.

The Hack Together Day had numerous exciting projects; the ASCL’s projects were less glamorous than most others but yielded really useful information, some of which has already been added to ASCL entries.

Our collective efforts went very well, despite a few worrisome moments along the way. The room our Special Session presentations were in had 98 seats; perhaps 90% were filled for these sessions, and there were people also standing in the room. The presenters/presentations were great and the discussions were lively, and more information about these sessions will be posted soon.

There was of course much much more to EWASS than our efforts; notable for those software-inclined were the astrometry, big data, and astroinformatics sessions and associated posters for all of these sessions. In all, an excellent conference!

Perspectives in Research Software Special Session at AAS 229

Special Session: Perspectives in Research Software: Education, Funding, Reproducibility, Citation, and Impact
10:00 am – 11:30 am
Grapevine 2

The Moore-Sloan Data Science Environment at NYU and the ASCL have organized a Special Session at January’s AAS. The session, Perspectives in Research Software: Education, Funding, Reproducibility, Citation, and Impact, will be moderated by Bruce Berriman (IPAC, Caltech/Astronomy Computing Today). The session will feature short presentations and will include a discussion period with the floor open for questions and comments, and maybe even a few answers, too. The topics and presenters are :

Tracy Teal (Data Carpentry), Software not as a service
Michael Hucka (Caltech), Finding the right wheel when you don’t want to reinvent it
Lior Shamir (LTU), Reproducibility and reusability of scientific software
Ivelina Momcheva (STScI), Funding research software development
Heather Piwowar (ImpactStory), Capturing the impact of software
David W. Hogg (NYU), The relationships between software publications and software systems
Alice Allen (ASCL), Update on research software citation efforts

That last speaker looks a wee bit dodgy, but the moderator and other panelists are aces! And you, software authors and users, are as always important participants in the discussion. I hope to see you there!

Licensing Astrophysics Codes session at AAS 225

On Tuesday, January 6, the ASCL, AAS Working Group on Astronomical Software (WGAS), and the Moore-Sloan Data Science Environment at NYU sponsored a special session on software licenses, with support from the AAS. This subject was suggested as a topic of interest in the Astrophysics Code Sharing II: The Sequel session at AAS 223.

Frossie Economou from the LSST and chair of the WGAS opened the session with a few words of welcome and stressed the importance of licensing. I gave a 90-second overview of the ASCL before turning the podium over to Alberto Accomazzi from NASA/Astronomy Data System (ADS), who introduced the panel of speakers and later moderated the open discussion (opening slides), after which Frossie again took the podium for some closing remarks. The panel of six speakers discussed different licenses and shared considerations that arise when choosing a license; they also covered institutional concerns about intellectual property, governmental restrictions on exporting codes, concerns about software beyond licensing, and information on how much software is licensed and characteristics of that software. The floor was then opened for discussion and questions.

photo of audience at licensing session

Discussion period moderated by Alberto Accomazzi

Some of the main points from each presentation are summarized below, with links to the slides used by the presenters.

    • Copy-left and Copy-right, Jacob VanderPlas (eScience institute, University of Washington)
      Jake extolled everyone to always license codes, as in the US, copyright law defaults to “all privileges retained” unless otherwise specified. He pointed out that “free software” can refer to the freedoms that are available to users of the software. He covered the major differences between BSD/MIT-style “permissive” licensing and GPL “sticky” licensing while acknowledging that the difference between them can be a contentious issue.
      slides (PDF)
    • University tech transfer perspective on software licensing, Laura L. Dorsey (Center for Commercialization, University of Washington)
      Universities care about software licenses for a variety of reasons, Laura stated, which can include limiting the university’s risk, respecting IP rights, complying with funding obligations, and retaining academic and research use rights. She also covered factors software authors may care about, among them receiving attribution, controlling the software, and making money. She reinforced the importance of licensing code and discussed the common components of a software license.
      slides (PDF)
    • Relicensing the Montage Image Mosaic Engine, G. Bruce Berriman (Infrared Processing and Analysis Center, Caltech)
      In last year’s Astrophysics Code Sharing session, Bruce had discussed the limitations of the Caltech license under which the code Montage was licensed; since then, Montage has been relicensed to a BSD 3-Clause License. Following on the heels of Laura’s discussion and serving as a case study for institutional concerns regarding software,  Bruce related the reasons for and concerns about the relicensing, and discussed working with the appropriate office at Caltech to bring about this change.
      slides (PDF)
    • Export Controls on Astrophysical Simulation Codes, Daniel Whalen (Institute for Theoretical Astrophysics, University of Heidelberg)
      image of presentation slide

      Restricted algorithms; image by Adam M. Jacobs

      Dan’s presentation covered some of the government issues that arise from research codes, including why certain codes fall under export controls; a primary reason is to prevent the development of nuclear weapons.Dan also brought up how foreign intelligence agencies collect information and what specific simulations are restricted, and stated that Federal rules are changing, but slowly.
      slides (PDF)

    • Why licensing is just the first step, Arfon M. Smith (GitHub Inc.)
      Arfon went beyond licensing in his presentation to discuss open source and open collaborations, and how GitHub delivers on a “theoretical promise of open source.” He shared statistics on the growth of collaborative coding using GitHub, and demonstrated how a collaborative coding process can work and pointed out that through this exposed process, community knowledge is increased and shared. He challenged the audience to contemplate the many reasons for releasing a project and to ask themselves what kind of project they want to create.
      slides (PDF)
    • Licenses in the wild, Daniel Foreman-Mackey (New York University)
      First, I have to note that Dan made it through 41 slides in just over the six minutes allotted for his talk, covering about seven slides/minute; I don’t know whether to be more impressed with his presentation skills or the audience’s information-intake abilities!

      17% of GitHub repositories examined are licensed

      Percentage of licensed GitHub repos; image by Arfon Smith

      After declaring that he knows nothing about licensing, Dan showed us, and how, that he knows plenty about mining data and extracting information from it. From his “random” selection of 1.6 million GitHub repositories, he noted with some glee that 63 languages are more popular on GitHub than IDL is, the number of repositories with licenses have increased since 2012 to 17%, and that only 28,972 of the 1.6 million mentioned the license in the README file. Dan also determined the popularity of various licenses overall and by language and shared that information as well.
      slides (PDF)

Open Discussion
After Dan’s presentation, Alberto Accomazzi opened the floor for discussion. Takeaway points included:

  • Discuss licensing with your institution; it’s likely there is an office/personnel devoted to deal with these issues
  • This office is likely very familiar with issues you bring to it, including who to refer you to when the issues are outside their purview
  • “Friends don’t let friends write their own licenses.” IOW, select an existing license rather than writing your own
  • License your code
  • Let others know how you want your code cited/acknowledged

My thanks to David W. Hogg, Kelle Cruz, Matt Turk, and Peter Teuben for work — which started last March! — on developing the session, to Alberto for his excellent moderating and to Frossie for opening and closing it. My thanks also to the wonderful Jake, Laura, Bruce, Dan W, Arfon, and Dan F-M for presenting at this session, and to the Moore-Sloan Data Science Environment at NYU and AAS for their sponsorship.

Many resources on licensing, including excellent posts by Jake and Bruce, can be found here.

Tweets from and about the code sharing session at AAS223

The code sharing crowd took over the AAS Twitter feed, it seems, during the Special Session on code sharing at AAS 223. Bottom up is the best way to read these, as the most recent tweet is on the top, and please note they aren’t strictly in order of occurrence and I likely missed some (there were so many!). I’m happy to add those I missed if someone tells me about them. Thanks to all those who tweeted throughout the session!

  1. ASCL@asclnet 10 Jan
    I keep a list of articles of possible interest to #astroCodeShare folks here: …. #aas223
  2. Nuria Lorente@NoTruerAlien 7 Jan
    @bruceberriman Absolutely, but NOT releasing code also comes at a price, which is often forgotten. #aas223 #astroCodeShare
  3. Zach Pace@zpacefromspace 7 Jan
    Just got finished with an awesome breakout session at #aas223 on code sharing. The moral: your code may be crap, but release it anyway!
  4. Nuria Lorente@NoTruerAlien 7 Jan
    Morin et al: Informative paper on Sw licensing for Scientist-Programmer. MT @augustmuench: #astroCodeShare #aas223
  5. Chrissy Madison@cmmadiso 7 Jan
    See. It happens! RT @bathompso: Pulling a @cmmadiso: my phone has 1% battery after the #astroCodeShare session. #AAS223
  6. Ben Thompson@bathompso 7 Jan
    Pulling a @cmmadiso: my phone has 1% battery after the #astroCodeShare session. #AAS223
  7. Adrian Price-Whelan@a_p_w 7 Jan
    RE: writing quick/dirty code to get papers out. “Weeks of programming can save you hours of planning.” #aas223
  8. Lucianne Walkowicz@shaka_lulu 7 Jan
    Hanisch: there should also be a prize for software, esp since Webber prize is for hardware only #aas223 #astroCodeShare
  9. August Muench@augustmuench 7 Jan
    .@AlexaVillaume note: that paper is for software. licensing of *data/papers* is distinct but VERY important thing. #aas223 #astroCodeShare
  10. Lucianne Walkowicz@shaka_lulu 7 Jan
    Licensing: BSD or MIT and forget about it- but we should discuss it more as community – @davidwhogg #astroCodeShare #aas223 cc @jonmccann
  11. Lucianne Walkowicz@shaka_lulu 7 Jan
    crap I had to answer an email and missed license discussion. Maybe check #astroCodeShare tag if someone else got it
  12. Christopher Hanley@chanley 7 Jan
    Projects should include a citations file in repo right next to LICENCE.txt and README. Make it easier to be cidted #astroCodeShare
  13. August Muench@augustmuench 7 Jan
    so @aaccomazzi brings us back to licensing: unlicenced code is the WORST. @davidwhogg echos this point #aas223 #astroCodeShare
  14. Lia Corrales@eblur27 7 Jan
    Hey #astroCodeShare, I still want to know how I should cite software I get through github. Could help with fluid contributor lists #aas223
  15. Lucianne Walkowicz@shaka_lulu 7 Jan
    Something to be gleaned from size of room v attendance v perceived necessary size of room & how code is valued. #aas223 #astroCodeShare
  16. Ben Cook@bacook17 7 Jan
    Reference in #aas223 code sharing session.
  17. Kelle Cruz@kellecruz 7 Jan
    lots of different ways to share code…but I really want to spend time and energy making it expected & common practice.
  18. Lucianne Walkowicz@shaka_lulu 7 Jan
    Comment in back: u make good science by making good investments- invest in quality code by encouraging code sharing #aas223 #astroCodeShare
  19. Kelle Cruz@kellecruz 7 Jan
    I care less about how we data & code share. tech will work itself out. I want to make it a *requirement* for funds and publications.
  20. Alexa Villaume@AlexaVillaume 7 Jan
    A mortifying story of a misplaced 2 in a program causing 8 years of research going down the drain. Share your code. It’ll be ok. #aas223
  21. Meredith Rawls@merrdiff 7 Jan
    .@kellecruz If it’s easy to share code & get credit, we’ll do it. Reminds me of this: … #aas223 #astroCodeShare
  22. Lucianne Walkowicz@shaka_lulu 7 Jan
    There are more ppl in this room than were in the Kepler session I attended yesterday. #aas223 #astroCodeShare
  23. August Muench@augustmuench 7 Jan
    225: Panel: MMMMMM Q: FMMFFFMM A: all M except 1 comment by F audience #astroCodeShare
  24. August Muench@augustmuench 7 Jan
    So @eteq is gonna drop the mic: papers have fixed author lists. software authorship if fluid and grows. Et tu, ADS? #astroCodeShare #aas223
  25. Lucianne Walkowicz@shaka_lulu 7 Jan
    Reply: if it goes on arXiv you can never update contributor list, so subsequent contributors don’t get credit #aas223 #astroCodeShare
  26. Lucianne Walkowicz@shaka_lulu 7 Jan
    Prsa: would be great if announcement of code went up on arXiv (I think they often do as release papers, e.g. emcee) #aas223 #astroCodeShare
  27. August Muench@augustmuench 7 Jan
    interesting point: Montage built under contract to NASA; astropy built by cats, hosted on a cat based website #astroCodeShare #aas223
  28. Meredith Rawls@merrdiff 7 Jan
    Andrej Prsa implores everyone in #astroCodeShare session to post code on astro-ph every time you submit a paper. #aas223
  29. Meredith Rawls@merrdiff 7 Jan
    My advisor has said not to “waste time” writing generalized code; contradicting this is troubling. Mixed messages. #aas223
  30. Lucianne Walkowicz@shaka_lulu 7 Jan
    Cost to sharing: making code useable by anyone req more time than just making it work for you then publishing w it #aas223 #astroCodeShare
  31. Meredith Rawls@merrdiff 7 Jan
    Recurring theme of how do I maximize research productivity and make my code useable for others? Not an easy Q. #AAS223 #astroCodeShare
  32. Lucianne Walkowicz@shaka_lulu 7 Jan
    That is, proprietary data sometimes equates to leverage- there is prob some analogy in code community- @davidwhogg #aas223 #astroCodeShare
  33. Lucianne Walkowicz@shaka_lulu 7 Jan
    panelists doubt ppl are being hired for a specific code as opposed to skill, but must be analogy w proprietary data #aas223 #astroCodeShare
  34. Lucianne Walkowicz@shaka_lulu 7 Jan
    Q: how do we reward ppl in ways that don’t req keeping code proprietary? As in, ppl get hired bc they have the code #aas223 #astroCodeShare
  35. Lia Corrales@eblur27 7 Jan
    Sad I’m missing #astroCodeShare, but reports of a packed room and massive twitter coverage are letting me stay comfy in COS session #aas223
  36. Lucianne Walkowicz@shaka_lulu 7 Jan
    Besides, every little thing you think no one else needs- *someone* will prob find it useful #aas223 #astroCodeShare
  37. Lucianne Walkowicz@shaka_lulu 7 Jan
    A: do what you need, if no one else needs it then that’s fine, you haven’t made anyone’s life worse – @davidwhogg #aas223 #astroCodeShare
  38. Lucianne Walkowicz@shaka_lulu 7 Jan
    Q: what’s the balance bt needing to make code work for yourself vs making it useful for everyone always? #aas223 #astroCodeShare
  39. Lucianne Walkowicz@shaka_lulu 7 Jan
    Comment in back: u make good science by making good investments- invest in quality code by encouraging code sharing #aas223 #astroCodeShare
  40. Lucianne Walkowicz@shaka_lulu 7 Jan
    Katz: not really, unlikely beyond a few years’ horizon at a time #aas223 #astroCodeShare
  41. Lucianne Walkowicz@shaka_lulu 7 Jan
    No long term stewardship of code like there is for results (i.e. pubs)- does NSF have plans for that? – @davidwhogg #aas223 #astroCodeShare
  42. Ben Thompson@bathompso 7 Jan
    . @kellecruz starting off the #astroCodeShare question session strong. Why no AAS reps here? #AAS223
  43. Lucianne Walkowicz@shaka_lulu 7 Jan
    Do you use other people’s codes? Do you modify them or use them as is? #astroCodeShare #aas223
  44. Lucianne Walkowicz@shaka_lulu 7 Jan
    Do you share code? If not, why not? #astroCodeShare #aas223
  45. Lucianne Walkowicz@shaka_lulu 7 Jan
    Benefits: perceived priority on work, visibility & good will, citations, bug-catching, and moral high ground – DWH #astroCodeShare #aas223
  46. Alexa Villaume@AlexaVillaume 7 Jan
    Releasing code establishes priority and good will. Benefit from bug catching. Also, you get to be smug. #aas223
  47. Laura Watkins@laurawatkins_ 7 Jan
    : if you’re not embarrassed by the code you released, you released it too late. #aas223
  48. Ian Paul Freeley@ianpaulfreeley 7 Jan
    If your not embarrassed by your code/website, you launched too late–Hogg #aas223
  49. Ben Thompson@bathompso 7 Jan
    If you’re not embarrassed by your code, you’re releasing it too late #AAS223 #astroCodeShare
  50. Meredith Rawls@merrdiff 7 Jan
    .@davidwhogg debunks cons to code sharing. Only real cost is email & support requests. He knows of NO example of being scooped. #aas223
  51. August Muench@augustmuench 7 Jan
    hogg: “if you’re not embarrassed by the code/website you put out there then you put it out there too late.” so good. #astroCodeShare #aas223
  52. Lucianne Walkowicz@shaka_lulu 7 Jan
    Cost: embarrassment! You know your code is crap, but if yr not embarrassed you released too late. –@davidwhogg #aas223 #astroCodeShare
  53. Lucianne Walkowicz@shaka_lulu 7 Jan
    Costs: getting scooped? @davidwhogg knows of no cases of scooping caused by *release of code* #astroCodeShare #aas223
  54. Lucianne Walkowicz@shaka_lulu 7 Jan
    All papers, grant writing, etc – not just code – are developed out in the open since 2005. – @davidwhogg #astroCodeShare #aas223
  55. Matthew Turk@powersoffour 7 Jan
    Not all good or new software is developed using github. Platforms should be transcended by applications. #astroCodeShare
  56. Lucianne Walkowicz@shaka_lulu 7 Jan
    And boom, @davidwhogg right on time. Also, who mic’d him? #astroCodeShare #aas223
  57. Laura Watkins@laurawatkins_ 7 Jan
    +1 MT @augustmuench “and this fact terrifies me because we have no idea collectively what sharing should look like. #astroCodeShare #aas223
  58. August Muench@augustmuench 7 Jan
    and this fact terrifies me because education — we have no idea collectively what sharing should look like. #astroCodeShare #aas223
  59. August Muench@augustmuench 7 Jan
    Who enforces software/data sharing in astronomy? YOU DO! WE DO! PEER REVIEW DOES! not nsf/nasa. #aas223 #astroCodeShare
  60. Lucianne Walkowicz@shaka_lulu 7 Jan
    Its UP TO YOU to include good data management plan as part of panel reviews. The community must enforce importance. #aas223 #astroCodeShare
  61. Lucianne Walkowicz@shaka_lulu 7 Jan
    Data management plans in NSF proposals are required to detail how results/data/software will be shared. – Katz #astroCodeShare #aas223
  62. Lucianne Walkowicz@shaka_lulu 7 Jan
    NSF policy for sharing research results: supposed to share not only the data and the results but the software #astroCodeShare #aas223
  63. Lucianne Walkowicz@shaka_lulu 7 Jan
    NSF does include “products” in addtn to pubs in bio sketches, but could be better abt following up on code release #AAS223 #astroCodeShare
  64. Lucianne Walkowicz@shaka_lulu 7 Jan
    Do we have policies that mandate code release in conjunction w publication or receipt of fed funds? #aas223 #astroCodeShare
  65. August Muench@augustmuench 7 Jan
    Software that enables all this new software: Github, Travis, Sphinx, Jenkins. #aas223 #astroCodeShare
  66. August Muench@augustmuench 7 Jan
    Agreed. RT @kellecruz: .@augustmuench we need to make data/code sharing requirements part of AAS journal policy. those two things. #aas223
  67. Ben Thompson@bathompso 7 Jan
    Testing code is an important part of code sharing. #aas224 session? #AAS223
  68. Kelle Cruz@kellecruz 7 Jan
    .@augustmuench we need to make data/code sharing requirements part of AAS journal policy. those two things. #aas223
  69. Lucianne Walkowicz@shaka_lulu 7 Jan
    If you build it, they will code – Tollerud #aas223 #astroCodeShare
  70. Lucianne Walkowicz@shaka_lulu 7 Jan
    Need infrastructure, a few software ppl to do housekeeping, let scientists do whatev & set expectations – Tollerud #astroCodeShare #aas223
  71. Lucianne Walkowicz@shaka_lulu 7 Jan
    Most ppl who have contributed code to AstroPY have never met each other – all via @github – Tollerud #astroCodeShare #aas223
  72. Ben Thompson@bathompso 7 Jan
    Almost 60 people (who have not met) have all worked together to build @astropy #astroCodeShare #AAS223
  73. Lucianne Walkowicz@shaka_lulu 7 Jan
    AstroPY: a python library for and by astronomers, developed by self-herding astronomers since 2011 – Tollerud #astroCodeShare #aas223
  74. August Muench@augustmuench 7 Jan
    I hoping that we see some cool diffs between the @astropy and montage *support* networks in the open discussion in #astroCodeShare #aas223
  75. August Muench@augustmuench 7 Jan
    . @merrdiff “research objects” is I think the new age terminology. #aas223 #astroCodeShare
  76. David Morrison@drmorr0 7 Jan
    Best advice I have: learn to use Git (or SVN, if you must), and use it for every single piece of code you write. #astroCodeShare
  77. Lucianne Walkowicz@shaka_lulu 7 Jan
    I have used this exact cat herding graphic in Erik Tollerud’s talk in an LSST talk hehe #aas223 #astroCodeShare
  78. August Muench@augustmuench 7 Jan
    the @astropy project — cat herding software development from @eteq at #astroCodeShare #aas223
  79. Ian Paul Freeley@ianpaulfreeley 7 Jan
    Damn it–tweets from code sharing session sounded cool, but I got here late and now crowd out the door. #aas223
  80. Alex Parker@Alex_Parker 7 Jan
    I’m nodding so vigorously at the #astroCodeShare tweets that I might need to ice my neck later.
  81. Dr Chris Tibbs@chris_tibbs 7 Jan
    Love the fact that my timeline is currently full of great tweets about code sharing and EPO #aas223
  82. Kelle Cruz@kellecruz 7 Jan
    ok, maybe 20% women in code sharing session but still disproportionately tweeting. #interesting
  83. Alexa Villaume@AlexaVillaume 7 Jan
    “I wrote my first fortran code when Apollo 12 was on the moon.” #aas223
  84. August Muench@augustmuench 7 Jan
    Decision to code Cloudy in C++ was partly motivated to use industry grade lang & give students real world job ops! #astroCodeShare #aas223
  85. Meredith Rawls@merrdiff 7 Jan
    Learning about the CLOUDY code, but speaker has no visuals 🙁 Jokes that the code can be opaque; “C++: write once, read never.” #aas223
  86. Lucianne Walkowicz@shaka_lulu 7 Jan
    Complaining astros aren’t comp scientists is like saying they shldn’t learn math bc they aren’t mathematicians #astroCodeShare #aas223
  87. Timothy Pickering@te_pickering 7 Jan
    ! RT @shaka_lulu: I’ll paraphrase @mjuric here: code is to modern astronomy what calculus once was. #aas223 #astroCodeShare
  88. Lucianne Walkowicz@shaka_lulu 7 Jan
    I’ll paraphrase @mjuric here: code is to modern astronomy what calculus once was. #aas223 #astroCodeShare
  89. Kelle Cruz@kellecruz 7 Jan
    could someone in the back of the code sharing session do a quick attendance & gender count? I’m in the front row…
  90. Jessica Lu@jlu_astro 7 Jan
    Tell me if you figure it out!
  91. Kelle Cruz@kellecruz 7 Jan
    code sharing room is packed! I’m curious what brought them all here…
  92. Lucianne Walkowicz@shaka_lulu 7 Jan
    Q: how much do you think we fail to educate our young researchers to write good code? #astroCodeShare #aas223
  93. Ben Thompson@bathompso 7 Jan
    Q on why students are not educated on how to write good code (or code at all!) #AAS223
    We have all failed here.
  94. August Muench@augustmuench 7 Jan
    The code base under question is Montage , dev’d & now volunteerly supported by IPAC scientists #aas223 #astroCodeShare
  95. August Muench@augustmuench 7 Jan
    “Releasing your code comes with a price” — @bruceberriman Hmm, let’s see if this pivots to the positive+solutions! #aas223 #astroCodeShare
  96. Lucianne Walkowicz@shaka_lulu 7 Jan
    Lastly, resist the pundit-technician divide. – Weiner #aas223 #astroCodeShare
  97. August Muench@augustmuench 7 Jan
    I completely agree with @cloud149: a lot of our concerns about sharing code are “pseudo” or hypothetical problems. #aas223 #astroCodeShare
  98. Michelle Collins@michelle_lmc 7 Jan
    We are failing to teach students how to write GOOD code in astronomy. Need to do better. Some programs in place, but not standard #aas223
  99. Laura Watkins@laurawatkins_ 7 Jan
    “Do we do enough to teach our researchers how to write good code?” No. Fundamental skills but so many are left to learn alone. #aas223
  100. Kelle Cruz@kellecruz 7 Jan
    really interesting that nearly 100% of the women in the code sharing session are tweeting…all 4 of us. #exaggerating
  101. Laura Watkins@laurawatkins_ 7 Jan
    Standing room only at the code sharing session. Apparently this is more popular than anticipated (this can only be a good thing)! #aas223
  102. Meredith Rawls@merrdiff 7 Jan
    Astrophysics code sharing session 225 at #aas223. Let’s stop re-inventing the wheel. Our hardware is built to last; why not software?
  103. Michelle Collins@michelle_lmc 7 Jan
    Oh, there are no women on the code sharing panel. Are we not sharing code? I’m currently not, but i’m here to learn how to #aas223
  104. Ben Thompson@bathompso 7 Jan
    Excited for the Astronomy Code Sharing session. Wondering what to do with all my research programs. #AAS223
  105. Erik Tollerud@eteq 7 Jan
    , Tues@2pm: talking about lesson’s from @astropy on how code can be shared, along side @owlice @davidwhogg @bruceberriman @cloud149
  106. Benjamin Weiner@cloud149 7 Jan
    My talk “Occupy Hard Drives” for code session Tues 2 pm #aas223 is here: @davidwhogg @bruceberriman @owlice @eteq
  107. ADASS@astroADASS 7 Jan
    Follow discussion on astronomy code sharing at the #aas223 meeting using #astroCodeShare hashtag.
  108. Benjamin Weiner@cloud149 7 Jan
    Tues 2pm #aas223 I aim to provoke on astro code sharing and why we don’t respect software with @davidwhogg @bruceberriman @owlice @eteq
  109. Astropy@astropy 6 Jan
    At the #aas223? Don’t miss Tuesday’s 14:00-15:30 session on code sharing – including a talk by @eteq about @astropy!

  110. David W. Hogg@davidwhogg 6 Jan
    Tues at 2 see @owlice Hanisch Teuben @cloud149 @bruceberriman Ferland Katz @eteq and me get all crazy about #code sharing at #aas223 in NH5

Astrophysics Code Sharing II: The Sequel at AAS 223

On Tuesday, January 7, the AAS Working Group on Astronomical Software (WGAS) and the ASCL sponsored a special session on code sharing as a follow up to the splinter meeting “Astrophysics Code Sharing?” held at AAS 221. We continue the dialogue for ways to improve the transparency and efficiency of research by sharing codes and to mitigate the negative aspects of releasing them.

Photograph of room session was held in, showing people in seats and standing in the back of the room

Even before the session began, it looked like there would be standing room only. Photo, Peter Teuben, used with permission

Before the session started, however, there were a few nerve-wracking moments;  weather- and Amtrak-related delays had one of the presenters arriving at AAS at 2:40 AM the day of the session rather than before lunch on Monday, and another getting to AAS after the session had started (!) but before his talk was to begin. So yes! There were minutes to spare!

The standing-room-only session was moderated by Peter Teuben of the University of Maryland and chairman of the ASCL Advisory Committee; Robert Hanisch, STScI, outgoing chair of the WGAS and also a member of the ASCL Advisory Committee, provided closing remarks. Those not in the room were not without news of what was being said in it, as there was much tweeting about the session (#aas223, #astroCodeShare).

Peter started the session by introducing the speakers (present or not) and explaining a bit how the session would work: code case studies would have 2-minute question periods for any clarifications or questions about the cases themselves, and other questions would be deferred until the open discussion period, which was approximately the latter half of the session.

A very brief summary of some main points of the sessions, along with their titles, presenters, and links to slides where applicable, is given here.

    • Occupy Hard Drives: Making your work more valuable by giving it away, Benjamin Weiner (University of Arizona)
      Ben pointed out that time spent writing software represents an enormous sunk cost that is, unfortunately, not viewed as doing real work, though writing software is part of doing science. He stated that widely-used software has enabled at least as much science as a new instrument would. He encouraged people to document their code for their own sake, release it without worrying about bugs or other potential issues in the software, and to write software methods papers for journals.
      slides (PDF)
    • Maintaining A User Community For The Montage Image Mosaic Toolkit, Bruce Berriman (Caltech)
      In this case study of Montage, Bruce stated that releasing software comes with a cost, but that it is still worth doing. Montage was developed under contract, and was designed for ease of maintenance, modularity, and sustainability from the beginning. It is maintained primarily through volunteer effort, and in part through collaborations, e.g., with the LSST EPO team. He said the Caltech license under which Montage is licensed does not allow users to redistribute modified code, nor can Montage be included in other distributions such as Redhat. He suggests coders consider licensing carefully.
      slides (PDF)
    • Cloudy – simulating the non-equilibrium microphysics of gas and dust, and its observed spectrum, Gary Ferland (University of Kentucky)
      Gary discussed Cloudy, which, with over three decades of use, is the most mature of the three codes covered in this session. The code is autonomous and self-aware, providing warnings about what might have gone wrong when things do go wrong. Though the user community is broad and participants in the summer schools that are held on the code have formed collaborations, a Yahoo! discussion forum for Cloudy has not been as successful as they had hoped. Cloudy was released as open access, with the most permissive license possible; Gary cited NSF as making this necessary since the code was developed with public grant funds. Students who work on the code get industry-standard programming experience, which is intended to help students gain employment after graduation.
      slides (PDF)
    • NSF Policies on Software and Data Sharing and their Implementation, Daniel Katz (National Science Foundation)
      Dan covered the NSF policies that govern software funded by the agency. Though some NSF panels are much more rigorous than others, it is expected that PIs will publish all significant findings, included data and software; he stated quite firmly that data include software according to the Government. He also said that it is up to the community via peer review panels to enforce these policies, that many core research programs don’t enforce this very well, and that the community determines what is and is not acceptable. This may be changing, however, as with an Office of Science and Technology Policy memo on open data, OMB policies are pushing harder on open access.
      slides (PDF)
    • The Astropy Project’s Self-Herding Cats Development Model, Erik Tollerud (Yale University)
      The newest of the three code projects highlighted is Astropy. Erik described the grass-roots effort to self-organize the now ~60 code base contributors, and that this arose out of a common goal: to streamline astronomy tools written in Python, as having eight different packages to do the same thing means that 7/8s of the effort was wasted effort. He stated that technology now exists that provides good support for such an effort, including GitHub to manage the processes of many developers, Travis for testing code, and Sphinx for documentation, which is written as the code is written. He pointed out that agreement on the problem was the key in getting the effort to come together and that consensus, guidelines, and expectations make it work.
      slides (PDF)
    • Costs and benefits of developing out in the open, David W. Hogg (New York University)
      David started out by saying that everything his group does is open —  all papers, grant proposals, comments, and codes — and has been since 2005, and that this was a pragmatic, not an ethical decision. He stated that the negatives others give for not releasing code — getting scooped, embarrassment, time, e-mail and support requests, licensing — are overplayed, and that since the public is paying for this, we should return the products we develop to them. He doesn’t know of a single case of someone’s getting scooped because he/she shared code. Rather, the benefits that sharing openly provides, establishing priority, visibility and goodwill, re-use and citations, feedback and bug-catching, and having the moral high ground, outweigh the overplayed negatives.
      slides (PDF)

After David’s presentation, Peter opened the floor for questions and discussion, and Kelle Cruz from Hunter College was ready! Kelle said that AAS should require code release and then asked whether anyone from the AAS journals was present. There was not.

Photo of slide with unneeded discussion questions on them

We didn’t really need to prompt discussion; there was plenty to talk about! Photo, Meredith Rawls, used with permission

Kelle then suggested to Daniel Katz that the NSF should take stronger role in enforcement. Dan said he will see what he can do to get astronomy reviewers training for what to look for, and that he already does this for his area. David Hogg said there aren’t any mechanisms for long-term stewardship of software and asked whether the NSF was looking at this. Dan said it is not at this time, and that the NSF generally avoids long-term commitment of funds.

Someone in the back of the room pointed out that protection of code can also lead to the protection of errors, told a sad anecdote to illustrate that point, and commented that code sharing fosters improvements in coding practice. In response to a question about whether it was worthwhile to share very specific code, David answered yes, just post it, that if it’s not useful to others, so what? But it just might be! And Benjamin Weiner suggested the code be put in GitHub.

Two questions came from someone else in the back of the room, one on whether export control restrictions (ITAR) would be changing; the second question relayed that PhD students write a code for their thesis but then protect it because, in their perception, the code makes them employable, and did the panel have anything to say about that? Erik Tollerud made the point that people are hired for the skills that went into creating the code, not for a particular code. David replied that he has seen this with data, that proprietary data does sometimes give someone leverage for employment. Dan answered the ITAR question by saying that changes in ITAR were probably not going to happen soon.

Another attendee asked about the cost of making code shareable, of what that cost is, and felt that the panelists had swept it under the rug. Ben replied that it’s a community problem, the community needs to reward it, and there needs to be a values change. In the meantime, put it out there anyway; clean it up if you can, but put it out. David agrees there are costs, but the benefits are more substantial than the costs. The cost is not very large and the upside is larger than the downside. Bruce thinks it is worth the effort to plan upfront; that will save time/money later on. This is harder if the code is not initially planned, but one should try to address this when possible.

Nuria Lorente, who was following the session from Australia through Twitter, tweeted that “NOT releasing code also comes at a price, which is often forgotten.”

Andrej Prsa from Villanova made a strong appeal to post code to arXiv; he stated that astro-ph should be open to other things beside preprints. Someone else pointed out that arXiv doesn’t necessarily agree. David said that he put the documentation for emcee, the MCMC hammer on arxiv and that gets cited. Erik made the point about additional contributors to a software development project such as Astropy don’t get credit if they are not on the author list on the paper uploaded to arXiv. Alberto Accomazzi from ADS mentioned that updating the author list on arXiv was a way to fix that and give others credit, even though that will not be reflected on ADS.

Someone commented on the need for some sort of code sharing infrastructure to help with sharing. David commented that he wants all flowers to bloom, but some flowers are more valuable than others. Erik said that better search engines over time will help, that Astropy is more findable because of better search engines and because more people now link to it. It was mentioned that with more code sharing, finding useful codes may become more difficult as the signal to noise ratio goes down.

Alberto Accomazzi brought up the uncertain provenance of code, code that does not have a license, and sometimes no author, attached to it, and stated that it is hard to deal with because it cannot be shared. This was echoed by David, who pointed out that a lack of a license for a code can prevent release. Bruce suggested a licensing workshop would be a good idea, and this idea got traction among attendees. The recent re-licensing of yt was brought up. Dan Katz looks specifically for licensing information when looking at proposal, and it’s clear to him that many people don’t know what they are doing on this and could use guidance. David suggested that people use BSD or MIT licenses if they know nothing about licensing.

Peter Teuben then brought the discussion to an end and turned the podium over to Robert Hanisch for closing remarks.

Session wrapup
Robert Hanisch reiterated that software sharing is fundamental to the dissemination and validation of research results, and though there are carrots and sticks for software sharing, the sticks are not very strong. He also pointed out that nothing within the funding agencies offers support for software development and that there is a disconnect between national policy and implementation. Journals at best only encourage code release, too; they do not require it. A sociological change is necessary; in the meantime, he hopes those attending will just put codes out there!  The benefits outweigh the costs.

He talked also about opportunity for change; as of Sunday January 5, the Working Group on Astronomical Software has Frossie Economou as its new chair, and that over the weekend, the Council of AAS had suggested that the WGAS be elevated from a Working Group to a Division within AAS. He had requested that the Council have the WGAS offer a prize specifically for software, and though the Council did not accept the idea upon presentation, Bob noted that a Division can award prizes independently. Having a Division focused on software will also provide more visibility for it, and on this hopeful note, the session ended.

… though the discussion continues…

My thoughts (just a few)
This is the fourth discussion session the ASCL has arranged; previous sessions include one at AAS 221 and two at the previous two ADASS meetings. (Links to materials or discussion from previous sessions are below.)

I was glad to hear several of the presenters say the concerns people have about releasing their codes are overplayed. I was particularly happy when David said that if people would only go ahead and release their imperfect software, other people would see that released codes are also imperfect and thus feel more emboldened to release their own imperfect work. Yes! Lose the fear, gain the codes! It really doesn’t need to be perfect; Nick Barnes, among others, have written eloquently, or amusingly, on this subject already. Astronomical software wants to be free; please release it, let it show!

It was hard for me to stay silent when the need for a code sharing infrastructure was mentioned, not because I disagree with the need — I believe the need is very great! — but because the ASCL is trying hard to help with that. I’ve looked at other similar efforts tried over the years, and either they have started, lived (usually briefly) and in one case, even flowered, and died, or they still exist but are mostly silent and their efforts in code sharing dormant. The ASCL has been around since 1999 and is indexed by ADS, and use of it has been increasing. It’s not perfect, but it does work and is actively growing.

I believe that science should be as transparent as possible, that code release (absent ITAR and other truly compelling reasons) even if only for examination, not reuse, is part of this transparency, and that ultimately, code release is better for code authors, especially if the astronomy community works together to make it better for them. Code sharing can make astronomy more efficient, too, which is especially important in the current financial climate.

Finally, I want to thank Peter for moderating the session, Bob for offering closing remarks, and the most excellent Ben, Bruce, Gary, Erik, Dan, and David for presenting at this session and also for not protesting even one time about the innumerable emails they received from me from May on. I also have to thank our wonderful volunteer whose name I did not get, alas, for her great work and for counting the 149 (!) attendees, the AAS for accepting the proposal in the first place, and the amazing people who sent this session literally around the world through their tweets. Thank you!

AAS 221 Astronomy Code Sharing? links
Omar Laurino joins Astronomy Code Sharing panel
Brief blog post
Astronomy Computing Today post
Slides used at meeting: Google Doc  PDF

ADASS XXIII (2013) links
Our eight questions
The eight questions that were discussed/links to discussion notes
Pre-print of proceedings paper

ADASS XXII (2012) links
Birds of a Feather session
Resources used/linked to for ADASS
Pre-print of proceedings paper