Category Archives: special sessions

Software in Astronomy Symposium Presentations, Part 6

This is the sixth in a series of posts on the six-session Software in Astronomy Symposium held on Wednesday and Thursday, April 3-4 at the 2018 EWASS/NAM meeting.

BLOCK 6: Software Publishing Special Interest Group Meeting
This meeting-within-a-meeting was an opportunity for journal editors, publishers, referees, abstract services, and others associated with research software publication to discuss how best to include research software in the scholarly record, improve the sustainability and reproducibility of research articles, and share information on issues and possible solutions. Representatives from Science, Nature-Springer, MNRAS, Oxford University Press, and AAS Journals were among the journals and publishers attending. As the session was open to all, researchers and software authors also attended. The agenda had three main items on it: journal software policies, ratings for numerical reproducibility, and improving instructions for authors and referees. The session was moderated by Rein Warmels (ESO, DE) and Alice Allen (ASCL, US).

After introductions by all in the room, the first agenda item, journal policies on software, was opened for discussion. Keith Smith, associate editor for astronomy and planetary science for Science, shared that his journal is requiring that software that enables research results be shared. Editors from other journals stated that this would probably not work for them, though they are sympathetic to the importance of research software transparency. Chris Lintott, lead editor for Instrumentation, Software, Laboratory Astrophysics, and Data for AAS Journals, said that he rewrote the Journals’ software policies on his first day at the job to require formal citation of software. Smith pointed out the expectation is different for data; people are much more open about sharing observational data, and A&A (may) require it. Warmels noted a difference between data and software; the quality of observational data from an observatory is known. This is not the case with software. We cannot know the quality of the unreleased code. Amruta Jaodand asked whether publishing houses have software reviewers. The various astronomical societies’ journals do not peer-review code; there are a few journals that do perform code review of various depths, such as the Journal of Open Source Software and Software X, both of which focus on research software across disciplines.

There was support for better software citation, but not for ratings of articles for numerical reproducibility. The idea of ratings for reproducibility led to a discussion about reproducibility itself and the issue of releasing software written for research. Adam Leary, senior publisher at Oxford University Press, said that the Journal of Biostatistics rates the reproducibility of its articles and that that journal has a reproducibility editor. The group discussed the workload this might put on reviewers along with other issues, which would slow down the review. But the real job would be for the authors! Brigitta Sipocz mentioned the need for a feedback loop, which triggered the question as to why one would want to put a lot of resources into reproducibility. Smith replied that there are cases in other sciences where whole bodies of work could not be reproduced! Warmels pointed out that a number of fake results were found by the community, not by reviewers.

Someone suggested pushing the community toward releasing software through funding councils. Lintott initially liked the idea, and stated that journals could enforce this by checking papers against funding/funders that require release. Allen found this is an intriguing suggestion. Additional discussion raised several issues with this approach. The impracticality of implementing the idea became obvious when considering the time and resources it would take and the complexity of funding, as well as varied requirements of a large number of funding organizations.

Smith’s “virtuous cycle” slide

The discussion then turned to reasons researchers do not release their software, with one advantage stated as, “If you don’t give me this funding, this research will not be done.” We have to change the way we argue for funding, then… “because as the only person who can do this, keeping my code private IS an advantage.” Jaodand mentioned that researchers get less credit for software than for research results. Smith replied that until we build up a virtuous cycle of code release, something he had mentioned in his presentation the previous day, the answer may be getting the credit system working first.

Another disincentive for code release mentioned was the possibility of someone running software incorrectly and then publishing “this code doesn’t work.” Lintott said that we should look for this and get data on it, so we can answer the question, “How often has this happened?” He also suggested looking for the positive cases, where release has been good for a developer or developer team, and provide this data to code authors.

The next agenda item was improving instructions for authors and referees on software citation and treatment. According to Smith, Science’s instructions were improved by rewiting them to accord with the Center for Open Science‘s Transparency and Openness Promotion (TOP) Guidelines. The guidelines are very helpful, and using them provides clear instructions. Greg Schwartz, data editor for AAS Journals, asked how we could better encourage authors to read instructions. Smith waggishly replied these instructions exist so journal editors can point to them. It was suggested that journals standardize their instructions not only to help authors out but also to discourage what was referred to as “research tourism.”  The TOP Guidelines were again brought up as a good tool to use for standardizing instructions. Someone asked about having a section for acknowledgements or statements for software. Smith pointed out the danger that some may think that section as a substitute for formal citation. Allen agreed that software should have formal citations, and also stated her appreciation for the Software section that AAS Journals have added to their papers. The ASCL has long been interested in seeing such a software listing in research articles (in addition to, not as a substitute for, formal citation of software). Warmels returned the discussion to the idea of standardization of instructions, asking whether this can be done. Journal representatives said the various journals do get together to standardize where they can, and are due to do so again.

The discussion migrated to openness in general. Among the suggestions for moving the discipline to be more open were to “Advertise your openness!” and to include a slide in your presentations that say your work is open and reproducible; this lets your peers know that you value openness, and can help others think about working more openly. The point was made to not rely on policing for open practices as resources aren’t available to do so. The role of education was brought up, too: Researchers need to be taught how to make their data and software open.

The final agenda item was to decide whether an on-going software publishing special interest group might be welcomed by those in the room; there was no support for this. Journals already have a method to share information amongst themselves and everyone is oversubscribed to meetings, groups, and conference calls. With that item settled, the meeting and Software in Astronomy Symposium concluded.

The ASCL thanks the Heidelberg Institute for Theoretical Studies for its generous ongoing support, which permitted two participants in this symposium to attend the EWASS/NAM meeting who would not have been able to do so without it.

Software in Astronomy Symposium Presentations, Part 5

This is the fifth in a series of posts on the six-session Software in Astronomy Symposium held on Wednesday and Thursday, April 3-4 at the 2018 EWASS/NAM meeting.

BLOCK 5: Machine Learning & Data Mining
Stephen Serjeant (Open University, UK) moderated the fifth session. This session presented different techniques to, for example, study noise in gravitational wave interferometers, select young stellar object candidates, and directly image exoplanets. David Cornu’s (UTINAM, FR) talk, titled Selection of Spitzer YSO candidates using deep learning classifier, included a short tutorial for creating an artificial neural network, showing how a small neuron takes input vectors and updates weights associated with them to understand anti- and co-relation between various factors. Similarly, Carlos Alberto Gomez Gonzalez (U Grenoble Alpes) showed how supervised machine learning can be used to detect exoplanets in his presentation Data science for direct imaging of exoplanets. Other talks in this session included Massimiliano Razzano (INFN, IT), Deep learning to study the noise in gravitational wave interferometers, Sebastian Turner (LJMU, UK) presenting on k-means clustering in galaxy feature data, and Emille Ishida (COIN, FR) presenting for Santiago Gonzalez Gaitan, with Spatial inference of astronomical datasets with INLA. Robert Lyon (UManchester, UK) finished this session with a presentation on Imbalanced learning in astronomy, and provides a Jupyter notebook containing a tutorial and examples. The presentations in this session were accessible even to those with no experience in data mining and machine learning, as the techniques used were explained quickly and well before moving on to how they enabled particular research.

Slides from this session

Selection of Spitzer YSO candidates using deep learning classifier by David Cornu (pdf)

Data science for direct imaging of exoplanets by Carlos Alberto Gomez Gonzalez (pdf)

k-means clustering in galaxy feature data by Seb Turner (pdf)

Spatial inference of astronomical datasets with INLA by Emille Ishida/Santiago Gonzalez Gaitan (pdf)

Imbalanced learning in astronomy by Rob Lyon (pptx)

Software in Astronomy Symposium Presentations, Part 4

This is the fourth in a series of posts on the six-session Software in Astronomy Symposium held on Wednesday and Thursday, April 3-4 at the 2018 EWASS/NAM meeting.
BLOCK 4: Open & Transparent Data Services
Astronomy leads most sciences in providing many open services, particularly data and ways to get access to and use data. This session, moderated by Andy Pollock (USheffield, UK), highlighted some of the new and ongoing services available to not just professional astronomers, but also to students and other interested parties. Debbie Baines (ESA, ES) opened the session with her presentation ESASky version 2: the next generation, which included a live demonstration of this incredible resource. ESASky allows searching for any astronomical object and offers viewing it in different wavelengths. It is very fast, too; Baines loaded images from Herschel, XMM, Chandra, and HST in her live demo. It offers relevant links to SIMBAD, such as to papers, and one can pull up research literature from ADS in ESASky, too. Rachael Ainsworth (UMelbourne, AU), originally scheduled to give the first presentation, followed Baines; her presentation, Open Science in Astronomy, gave an overview of openness in astronomy. She pointed out that astronomy is better at open science than many other fields, crediting, among other services, arXiv and GalaxyZoo. She covered some of the challenges and shared additional resources that support, encourage, or make open science possible. Jorge Palacios (IFAE, ES), in his talk Astronomy in a Big Data platform, discussed two services, Cosmohub, a web portal for interactive exploration and distribution of massive cosmological data, and SciPIC (Scientific pipeline at PIC), software for generating synthetic galaxy catalogs using DM simulations. In An Interactive Sky Map based on the Byurakan Plate Archive, Gor Mikayelyan (BAO, AM) shared that the NAS RA Byurakan Astrophysical Observatory in Armenia is making its plate archive, covering 1947 -1991, available online in the BAO Observational Database, which will have the ability to search and select and will allow downloading the plates in different formats. A sky map will show where plates are available. After the last two talks, The ASI Cosmic Ray Database for charged particles data by Valeria Di Felice (SSDC/INFN, IT) and Using XML and semantic technologies in astroinformatics to manage data by Guy Beech (UHuddersfield, UK), Pollock opened the floor for discussion and asked whether people could do what they’d like to do as efficiently as they’d like. One answer from the audience was no, because the data are heterogeneous and different ways are required to access them. One pointed observation arising from the back-and-forth was that “radio astronomy is decades behind” in terms of software services, with “thousands of different file formats processed by thousands of different programs”.

Slides from this session

Open Science in Astronomy by Rachael Ainsworth (pdf)

Astronomy in a Big Data platform by Jorge Palacios (pdf)

An Interactive Sky Map based on the Byurakan Plate Archive by Gor Mikayelyan (pdf) | Text (pdf)

The ASI Cosmic Ray Database for charged particles data by Valeria Di Felice (pdf)

Using XML and semantic technologies in astroinformatics to manage data by Guy Beech (pdf) | Paper (pdf)

Software in Astronomy Symposium Presentations, Part 3

This is the third in a series of posts on the six-session Software in Astronomy Symposium held on Wednesday and Thursday, April 3-4 at the 2018 EWASS/NAM meeting.

BLOCK 3: Software packages for research
Amruta Jaodand
(ASTRON, NL) moderated the third session of the Symposium, which featured talks on seven different software packages. The first presentation was by Sergio Martin (ESO, CL), on MADCUBA and SLIM: A lightweight software package for datacube handling and spectral line analysis. These software packages offer features for easier use of the ImageJ image processing framework; this framework is widely used in other disciplines for multidimentional images. SLIM provides a synthetic spectra generation and automatic fit to observed spectra; MADCUBA provides an interface to handle and manipulate multiple datacubes, process large datasets quickly, and is scriptable. Alex Hamilton (UHull, UK) presented SunPy the Open Source Solar Physics Library, giving an introduction to the package, which collects, manages, and analyzes data from many solar data sources, and sharing information on its new features and capabilities.

As one of this session’s presenters was doubled-booked, we made adjustments to the order of presentations, so The next-generation cosmological code SWIFT was the next talk; this was given by Matthieu Schaller (LeidenU, NL). This package, due for release this summer, uses task-based parallelism for intra-node parallelization; testing has demonstrated that it is more than 30x faster than the code Gadget on representative cosmological problems while using fewer resources. The next talk had generated a lot of interest, as it is not a usual astronomy conference offering. Maisie Rashman (LJMU, UK) spoke about Developing and applying astronomical software for novel use in conservation biology. The team Rashman works on has developed a pipeline using astronomy techniques to identify and track animals; their goal is to create a fully automated system for species identification, population tracking, and combating poaching using drones. After Rashman, Shane Maloney (Trinity College, IE) presented xrayvision – a collection of image reconstructions methods for X-ray visibility observations. The xrayvision package is built atop other packages, including SunPy, and fills a need for an open-source solution for these observations. One of the project’s goals is to provide access to people in poorer countries to software for solar physics. The cleverly-named software package PampelMuse was presented by its author Sebastian Kamann (LJMU, UK) in his talk Crowded-field 3D spectroscopy with PampelMuse. This Python software performs PSF deblending on the integral-field data; several thousand sources can be deblended simultaneously. The final talk of this packed and fast-paced session was by Matteo Bachetti (INAF, IT) on Stingray, HENDRICS and Dave: Spectral Timing for all. Stingray is an AstroPy affiliated package that merges timing and spectral analysis for X-ray spectral timing. The idea for spectral timing arose in a meeting at Leiden University two years ago, where a team formed and started working on the project; the code is under active development. By combining Stingray with the HENDRICS (for shell scripting) and Dave codes (a GUI atop Stingray), the team will provide the community with a Python API (for the brave), a GUI to ease the learning curve, and shell scripting capabilities for batch processing, for advanced spectral timing with a correct statistical framework.

Slides from this session

MADCUBA and SLIM: A lightweight software package for datacube handling and spectral line analysis by Sergio Martin (pdf)

The next-generation cosmological code SWIFT by Mattheiu Schaller (pdf)

Stingray, HENDRICS and Dave: Spectral Timing for all by Matteo Bachetti (pdf)

Software in Astronomy Symposium Presentations, Part 2

This is the second in a series of posts on the six-session Software in Astronomy Symposium held on Wednesday and Thursday, April 3-4 at the 2018 EWASS/NAM meeting.

BLOCK 2: Software publishing, impact, & credit
This session focused on using the available infrastructure to better reward software authors and ways to count these valuable research objects. The software contributions that enable much of the results in astronomy are often not recognized, nor considered for reward or promotion. Unlike most of the other sessions in this Symposium, this session had only three short presentations and devoted the rest of the time to an open discussion and Q&A. The session and discussion period was moderated by Rein Warmels (ESO, DE).

Slide from Bianco’s presentation

Federica Bianco (NYU, US) presented Understanding the Impact of Your Research Software to open the session. She stated that one should always cite software used in research, but “it’s not always obvious how.” She discussed a finding of Howison and Bullard (2015) in their research on software citation in biology articles: open source software is cited more informally than proprietary software. This means that software authors are not accruing credit for their contributions in a way that academia rewards: formal citations. Bianco mentioned Force11, which has published software citation principles, and the now-completed Depsy project, which sought to provide not only citation information on software but also to measure the impact and use of code through other statistics, such as downloads, number of contributors and number of projects reusing the software. Among Bianco’s suggestions for fostering good citation practices for one’s own software were to get a DOI for it and give users instructions on how the software should be cited.

One of Smith’s slides

Keith Smith (Science, UK) spoke on Citation of data and software in astronomy: A journal editor’s perspective. Smith said that most scientific advances have their base on previous work, which requires reproducibility. Citations not only enhance reproducibility, they also assign credit. He provided guidelines for citing data and software, sharing bad, better, and good examples, and spoke of a virtuous cycle that will increase reproducibility in addition to the sharing of software and data. Smith mentioned the Center for Open Science Transparency and Openness Promotion guidelines, a policy framework for journals that was developed with journal and community input, and noted that though over 5,000 journals have signed onto them, none of the major astronomy journals have done so. Science’s policies require proper citation of data and software and release of data and software upon publication. Looking forward, Smith sees data and software citation becoming more common, as could be seen with a graph from the ASCL’s dashboard showing the increasing number of citations to its entries, and stated that journals have a role in improving reproducibility and proper citations through policies and editor and referee awareness of changing community standards.

The last presentation of this session was by Alice Allen (ASCL, US), who spoke on Receiving Credit for Research Software. She discussed recent changes in astronomy and in other disciplines that make recognizing the contributions of software authors easier. These changes include new journals, both astro-specific and with a broader focus, specifically for software, policy changes for existing journals, and community resources; these resources include collaborative coding sites such as Bitbucket and GitHub and archival resources such as Figshare and Zenodo. Existing services such as the ASCL have been given new life and are growing; in its Next Generation project, arXiv is improving its support for linking data and code to research. Software citation is captured/tracked/counted by indexers such as ADS, Web of Science, and Google Scholar. Broader efforts to improve reproducibility, citation,

Rein Warmels moderating

and credit, such as CodeMeta, Force11, WSSSPE, the FAIR principles, and DataCite involve those from many disciplines; the sharing of ideas influences not just those involved in the efforts, but has a greater reach with their aspirational and practical goals and guidelines. Allen shared steps code authors can take to increase the probability of having their software cited correctly and steps researchers can take to improve their articles by including citations for the computational methods that enabled their research, and provided a link to resources mentioned in her talk.

Rein Warmels then opened the floor for discussion. Someone asked whether GitHub would “be there forever?” The point was made that GitHub is not intended to be an archive, and that other services are, so use them to archive your code. On whether to release software, Smith stated that even horrible to read code is better than no code: “it’s hard, painful, and you may hate someone forever… but that’s better than nothing.” Science can provide software information as a supplement, so that is one way to ensure your software is available to support your research findings. In discussing software citations, Allen pointed out that ADS has not been able to automatically track citations to Zenedo DOIs, though that is expected to change soon. The issue of what exactly to cite also came up – should one cite all the dependencies needed to run a particular research code? Neither astronomy nor other disciplines have a way to handle this; at this time, the recommendation is to cite the research software you use well so others will know what your work relied on, and leave it to the software sites to identify the dependencies.

Slides from this session

Understanding the Impact of Your Research Software by Federica Bianco (pdf)

Citation of data and software in astronomy: A journal editor’s perspective_by Keith Smith (pdf)

Receiving Credit for Research Software by Alice Allen (pdf)

Software in Astronomy Symposium Presentations (2018 EWASS/NAM)

This is the first in a series of posts on the six-session Software in Astronomy Symposium held on Wednesday and Thursday, April 3-4 at the 2018 EWASS/NAM meeting. Each of the six sessions focused on a different aspect of research software, covering not only specific software packages, but also computational techniques used in data mining and machine learning, open services, software development training and techniques, and getting credit and citations for computational methods. Several sessions included a free-form period in which participants could ask questions, discuss issues, and share information. The last session of the Symposium was a lively moderated discussion among attendees with particular interest in software publishing.

BLOCK 1: Software engineering and sustainability, education for better software, & the ecosystem around Python in astronomy
The first session set the stage for the Symposium, featuring a variety of topics of importance when discussing astronomy research software. Alice Allen (ASCL, US) moderated the session. In the inaugural talk, Software Engineers as Partners in Astronomy Software Development, John Wenskovitch (Virginia Tech, US) opened his presentation with a quote by computer scientist and professor Carole Goble, stating that software is “the most prevalent of all the instruments used in modern science.” This was reiterated by others throughout the symposium. Wenskovitch provided statistics on software use and development activities by academics, among these that 92% of academics use software and 38% spend at least 20% of their time developing software. Research software engineers (RSE) provide guidance to researchers on software engineering and encourage the use of tools that can save academics time and effort in their development efforts. Wenskovitch suggests identifying and using the strengths of each, with the researcher bringing domain knowledge and expertise on the research itself, and the RSE bringing development experiendocument all the things!ce and software engineering expertise. He provided suggestions for ensuring a fruitful partnership; these include using version control, scheduling time for regular and frequent communication, having a prioritized feature list, testing the code thoroughly using unit, regression, and usability tests, and documenting everything.

photo of Mark WilkinsonMark Wilkinson (DiRAC HPC Facility, UK) spoke next, presenting Research Software Engineering – the DiRAC facility experience. The science requirements for DiRAC demand a 10-40fold increase in computing power to stay competitive, and this increase cannot be delivered solely by hardware. Software vectorization and code efficiency is vital, and RSEs are increasingly important to help with, for example, code profiling, optimization, and porting. DiRAC’s three full-time RSEs are embedded in teams, their time allocated through a peer-review process. Wilkinson showed that the use of RSEs has paid off well for DiRAC, with, for one project, a factor 10 speed-up by optimizing a particular code. The focus on software engineering continued with a talk on Software Engineering Training for Researchers delivered by David Perez-Suarez (UCL, UK). He presented information he had gathered by conducting a quick survey to learn, among other things, what software development training researchers had gotten. His recommendations for training include running or attending training taught by The Carpentries, asking that training be conducted in conjunction with a large conference, such as the  American Astronomical Society has been doing for several years, checking to see what software training might be offered by your university, creating your own study group, and contributing to an open source project. James Nightingale (DurhamU, UK) presented a very interesting talk on Test-driven Development in Astronomy. He convinced many in the room that using this technique for developing software will result in better software and less aggravation when coding. He stressed that test-driven development (TDD) is not a testing process, but a development process, and that the code coming out fully tested is a bonus. With TDD, the first task is not to write code, but to write a unit test and then run it to ensure it fails. Only after that do you write the code, and then test it. Through refactoring and testing code, you get instant feedback on whether the code’s functionality has changed, and code design becomes part of the development cycle.

The session then moved on to software sustainability with Bruce Berriman’s (Caltech/IPAC-NExScI, US) talk on Sustaining The Montage Image Mosaic Engine Since 2002. Montage has become increasingly robust and versatile over the years, is embedded in various archives and processing environments, and has been used in other disciplines as well as in astronomy. It has been cited more in information technology literature than in astronomy literature, though uptake of Montage was initially slow. Berriman made the point that design drives sustainability; all Montage releases inherit the design, and each module performs one task. He advocates listening to users and learning from their experiences, and shared his adage that “the grumpier the user, the more valuable the suggestions.”

The last two talks of this first session focused on Python, and covered the growing use of this language in astronomy, the reasons for this growth, the support that is available for the language, and information on one very popular package written in Python. Amruta Jaodand (ASTRON, NL) presented A Walk Through Python Ecosystem, starting with its early development in 1989 by Guido van Rossum at the University of Amsterdam. The advantages of Python include simplicity and natural flow and an extensive, powerful standard library. Strengths of the language include the development of scientific, numerical, and statistical packages and its Python Package Index (PyPi), which enables module and package sharing. Jaodand shared some of the learning materials available for Python, including python4astronomers, and also a lovely Easter egg about Python that is too long to include here and is worth reading. One of the most important astronomy packages is AstroPy, and Jaodand’s talk was followed by The Astropy Project: A community Python library and ecosystem of astronomy packages, presented by Brigitta Sipocz (AstroPy, UK). AstroPy provides software for many common astronomy needs; in addition to the core library, there are many affiliated packages. All of these packages adhere to coding, testing and documentation standards that have been developed by the AstroPy coding community. Sipocz also discussed the community, with members of the team having one or more of its many roles. The number of collaborators continues to grow, and the community welcomes new members, and labels packages that are particularly friendly for a new contributor to work on.

Slides from this session

Software Engineers as Partners in Astronomy Software Development by John Wenskovitch (PDF)

Research Software Engineering – the DiRAC facility experience by Mark Wilkinson (pdf)

Sustaining The Montage Image Mosaic Engine Since 2002 by Bruce Berriman (pdf)

Software Engineering Training for Researchers by David Perez-Suarez (Google doc) | blog post

Test-driven Development in Astronomy by James Nightingale (pdf)

A Walk Through Python Ecosystem by Amruta Jaodand (pdf)

EWASS/NAM Software in Astronomy Symposium

The EWASS/NAM Software in Astronomy Symposium gets underway at 9:00 AM today in Room 11A of the Liverpool ACC. This six-session Symposium includes presentations on:

  • Software engineering and sustainability, education for better software, and the ecosystem around Python in astronomy (Wednesday, 9:00 – 10:30 AM)
  • Software publishing, impact, and credit (Wednesday, 2:30 – 4:00 PM)
  • Software packages for research (Wednesday, 4:30 – 6:00 PM)
  • Open and Transparent Data Services (Thursday, 9:00 – 10:30 AM)
  • Machine Learning and Data Mining (Thursday, 2:30 – 4:00 PM)

The last session of the Symposium is a Software Publishing Special Interest Group meeting, and will take place on Thursday from 4:30 to 6:00 PM.

For more information on this session, including abstracts, check the interactive guide for Symposium S6a – S6f. See you there!

Report on the Astronomy Software Publishing Special Session at AAS231

On Thursday, January 11, the Astrophysics Source Code Library (ASCL) and Astronomical Data Group at the Flatiron Institute organized a Special Session at the 231st AAS meeting in National Harbor, MD on Astronomy Software Publishing: Community Roles and Services, the sixth in a series of software-focused sessions that the ASCL, sometimes with others, has organized at AAS meetings.

"Really glad to see software article publication and citation getting attention at the #AAS231 meeting. Great articles like Daniel Foreman-Mackey's "emcee: The MCMC Hammer in PASP is a perfect example of a highly-referenced software article."Peter Teuben from the University of Maryland and chair of the ASCL’s Advisory Committee) opened the session with a few words about the use of software in research articles. He outlined the layout of the session. A talk by Matteo Cantiello set the scene on how we have reached the point where we are now. Four presentations by representatives from different journals presented their policies on software publication followed Cantiello’s talk, and they were followed by presentations by representatives of others with roles in publishing software: the software author, the data editor, the ADS and the ASCL. The floor was then opened for discussion and Q&A. Teuben moderated the discussion, and at the end of it, turned the podium over to Robert Nemiroff from Michigan Technological University, and a founder of the ASCL, for a summary and closing remarks.

Presentations
Some of the main points from each presentation are summarized below; the titles of each are links to the slides used by the presenters.

  • The Evolution of Software Publication in Astronomy, Matteo Cantiello (Flatiron Institute)
    Cantiello states that the complexity of astrophysics requires computationally intensive models, making astronomy a digital science, and that astronomers have a rich computational environment available, allowing them to easily version, share, and deploy astronomy software. Reproducibility paradoxDespite this, software is often not shared, resulting in a reproducibility paradox: astronomers use computation to provide precise, accurate results, but research has become less transparent with the increase in the use of computational methods. Adding external links to papers to link to software is not a reliable solution to software sharing because of link rot. Formats have changed very little in the last 400 years; despite progress both technologically and socially, the format of papers is still largely the same. He stated that astronomy now has an opportunity to rethink scientific papers as research repositories, with executable objects containing narrative, figures, data, and code.
  • Software papers and citation in the AAS Journals, Chris Lintott (AAS Journals)
    The AAS journals policy on software until recently was set in 1964, which stated that the “need for communication between astronomers interested in computation is already supplied by associations of users of automated computing machines.” The AAS journals changed their policies at the beginning of 2016, and recognized that if novel code is important to published research then it is likely appropriate to describe it in such a paper. Papers can be short, descriptive, and need not include research resultsAAS journals are interested in disclosing software in a form that is currently recognized: the research article, so now allow short papers on code that can be short, descriptive, and do not need to include scientific results. AAS formally recommends open source licensing but does not require it. AAS journals ask people to cite the software paper, as this is the currency the field cares about and also ask people to cite the code. In addition, they request people use the \software{} tag to create a software section in a paper; this is similar to the \facilities{} tag already in use. AAS Publishing continues to think about how to improve, and are introducing the concept of ‘living’ papers, which can be updated with new sections and expanded author lists, so software authors don’t need to publish a new paper to give credit to software authors who have contributed to a new version of the software. Lintott encouraged those interested in living papers to contact him.
  • Software policies and guidelines at Nature, Leslie J. Sage (Nature)
    First, Sage explained the context in which Nature‘s policy is created: Nature is driven by biologists, who live in a very different world from astronomers. Unlike astronomers, biologists live in Windows world. Right now, two journals, Nature Methods and Nature Biotech, require code to be made available, and there are ongoing discussions about whether Nature should do this for other journals. There are formidable problems because of the issue of very specialized code, for example, code that is optimized to compile on a particular Beowulf cluster that may not compile anywhere else. There will be a call for public comment, and Sage hopes astronomers will provide input that is useful for astronomers within that context. Sage raised a number of points that warrant public discussion, such as a preference voiced by some to see detailed descriptions of the algorithms used rather than having the scripts published. Another point to consider for input is that though a lot of software has been made publicly available, all software is written with certain constraints and boundary conditions; some people not aware of these constraints and conditions may drive the software beyond those limits; this raises the question as to whether the results are physically meaningful.
  • SpringerNature data and software policies for astrophysics journals, Ramon Khanna (Springer)
    Springer is encouraging authors to take care of transparency and reproducibility of their results presented in articles, allowing them to append relevant information on source code or the full code in an appendix of the paper; authors can also append the full code, or use other methods to provide this information, such as alternative repositories (e.g., CDS, ASCL, Figshare), and making this information available. They would like the full data and code available. Khanna acknowledged some challenges, including that authors are often not willing to share their software and/or data, editors are often not willing or at least not determined enough to execute policy, and citation standards are unclear. One of the questions is how to execute this policy in the face of unwillingness from authors, editors, reviewers, etc.Questions arise as to how software can be peer reviewed; this would require standards for documentation,  presenting how the results were obtained, making data and software available, and for reviewing the source code itself. How can referees handle this effort? Khanna pointed out that in a field as advanced as astronomy is, and already having some standards and domain resources such as archives, it’s not so much the publisher that should drive new standards, but the community itself.
  • Journal of Open Source Software (JOSS): Design and first-year review, Arfon M. Smith (STScI/JOSS)
    Smith stated that he created JOSS accidentally, from frustration about the overhead of publishing papers about software, and acknowledged that software papers are a hack of the current system to provide a citable, creditable research object for software.  JOSS (http://joss.theoj.org/) seeks to improve the quality of software; its peer review process is almost entirely about the software that’s submitted, and includes making sure the documentation is sufficiently fleshed out, that the package includes automated tests, and that the software has an open source license so can be reused. Smith said it should take about an hour to write a one-page paper for JOSS for those with a well set up repository for their code. The reviews are public on Github and accepted submissions appear on the JOSS site, which has published 200 papers online.
  • Lessons Learned through the Development and Publication of AstroImageJ, Karen Collins (Center for Astrophysics)
    Collins discussed her experience with publishing her software AstroImageJ, a data reduction and image display interface with analysis capabilities specialized for time series differential photometry. She developed the code over several years to support her research. She initially had no intention of releasing the code to the public, but her collaborators saw her plots and graphs and asked to use the software, which was posted to the university’s website to give team members access to it. AstroImageJ lessons learnedShe found her fellow KELT-FUN team members were an excellent focus group; they provided great feedback on the software before it was published, enabling her to add useful features to the software. Results using AstroImageJ started appearing in journals; she registered the software with the ASCL to give it a citable reference, and as usage (and support tasks) grew, she and others working on the code decided to submit a paper to the Astronomical Journal (AJ) to provide good exposure to the potential userbase for the software. This resulted in about 4K downloads of the software in the first year, and the paper is listed 4th on AJ’s most read list. Among the lessons learned in publishing AstroImageJ are to specify how your code is licensed and how it should be cited, make the source code easily accessible, and provide easy way to install and update the software.
  • The roles of the AAS Journals’ Data Editors, August Muench (AAS Journals)
    Muench covered the data editors’ workflow for all submitted manuscripts. A quick review of 60%-90% of all submitted manuscripts is performed, with scripts run on the manuscripts to identify references to code by looking for such things as Github repositories to see whether  their citations need to be reviewed. The editors make notes on the software, data, and figures for review by a scientific editor or the author with recommendations for improving citations for these research artifacts. A subset of accepted articles, 15-20%, undergo a more rigorous post-acceptance data review; this includes a review of tabular data, figures, and interactive elements in addition to software. If necessary, the data editors request that authors acquire DOIs or get preferred citati"People recognize software via plots (and other fingerprints). Make sure you cite the code. I still recognize plots made with PAW."ons for the software used in the research. Muench mentioned that he uses ten keywords in his scripts to identify software, and ends up with a surprising number of articles that do not mention code at all. He stated that part of a data editor’s role to improve software and data citation is educating authors.
  • The role of the ADS in software discovery and citation, Alberto Accomazzi (NASA Astrophysics Data System)
    Accomazzi described what ADS does to promote software discovery and citation, but first he shared ADS’s traditional core responsibilities: to discover content, typically science papers, related to astronomy. Some years ago, the capability to track citations was introduced. As the expectations of community have evolved, so have ADS’s policies, moving from ingesting records about scientific papers to records about scholarly works, including data catalogs, observing proposals, and other artifacts such as software. They have also evolved from tracking citations to articles to citations to tracking citations to scholarly content. How ADS awards citationsADS has an interest in enabling linking so users can easily and uniquely identify the software that was used. Accomazzi covered how ADS ingestion works; for content to be considered for inclusion in ADS, it must be scholarly, related to astronomy, and published formally — not just on a website, but following an explicit editorial process. He also discussed how citations are tracked and what ADS needs to count a citation, going through several examples of what does and does not work for citation. The bottom line for software is to cite it by using a formal citation and a unique identifier; a URL to a website or a DOI in a footnote are not captured as citations. ASCL, JOSS, and Zenodo are ways software can get a persistent identifier to use in a formal citation, and these citations can be tracked by ADS. Accomazzi also discussed how software may have several records in ADS, and that in the future, these records will be crosslinked, as will different versions of a software package so that eventually, ADS can provide cumulative metrics for all different versions of that software product, and like all citation data, this information will become publicly available through an API.
  • The Astrophysics Source Code Library: Supporting software publication and citation, Alice Allen (ASCL/UMD)
    Allen gave a brief overview of what the ASCL is, and stated that though entries in this citable online registry usually point to a software package’s download it, the ASCL can and does serve as a repository for those authors who want to deposit an archive file for their code. The ASCL assigns a DOI to software that it stores. She covered the three main reasons the ASCL exists: to make research more transparent, to improve communication about research computations, and to disseminate software of utility to others. Allen acknowledged that though there is software that might be useful to astronomy, the ASCL focuses on that which has been used in refereed research or submitted for refereeing, this to support the research record. ASCL editors take an active approach in looking for software in research papers and registering them; authors are encouraged to submit their own software, too, and submission by author increased 23% in 2017 over 2016. The ASCL supports software publication and citation in a number of ways, including providing a citation avenue for software and listing preferred citation information in ASCL entries. The ASCL has been online since 1999; it supports the Force11 software citation principles and was a party to  developing them. It was also party to a Dagstuhl Manifesto, another cross-disciplinary effort  that focused on steps members of a research community can take on their own. Among these steps is citing software properly — in a trackable way — and when reviewing a paper, ensuring that it cites the software used in the research.

Discussion
After the presentations, Teuben commented that he thought journals could do a better job in instructing referees about software, to identify when code is involved in research and insist on citations to it. He hoped the discussion would touch on this, and then opened the floor to all. People interested in software sustainability might want to follow @si2urssi (urssi.us) which is working to plan a US research software systainability institute - a first workshop will likely be in AprilDiscussion was lively and may be covered in more depth in a future post, but some of the major points were:

  • There’s still fear about releasing software, still resistance to doing so
  • Science is all about reproducibility; it’s not science if it’s not reproducible
  • Who should push for greater openness is an open question, with some wanting journals to do this, and others feeling it’s up to the astronomy community — us! — to enforce the standards we want
  • Astronomers are often not trained in software engineering techniques; greater education in this area would be helpful

"If software developers were well funded, ie would be easier to get people to share their code."

Teuben brought the discussion to an end and turned the floor over to Robert Nemiroff (Michigan Technological University), who briefly summarized the presentations and discussion and closed the session.

My thanks to David W. Hogg and Peter Teuben for work on developing the session, to Peter for his excellent moderating, to Robert for closing the session, and for PW Ryan for serving as scribe. My thanks to Matteo, Chris, Leslie, Ramon, Arfon, Karen, Gus, and Alberto for their excellent presentations and participation, to the Astronomical Data Group at the Flatiron Institute for partnering with the ASCL, and to the Heidelberg Institute for Theoretical Studies, the University of Maryland College Park, and Michigan Technological University for supporting the ASCL.

Slides from Astronomy Software Publishing: Community Roles and Services

THURSDAY, 11 JANUARY 2018
Special Session: Astronomy Software Publishing: Community Roles and Services
10:00 am – 11:30 am
National Harbor 2

The Astrophysics Source Code Library (ASCL) and Astronomical Data Group at the Flatiron Institute organized a Special Session at the 231st AAS meeting in National Harbor, MD on Astronomy Software Publishing: Community Roles and Services. Click on a talk’s title to download its slides.


Matteo Cantiello (Flatiron Institute), The Evolution of Software Publication in Astronomy
Chris Lintott (AAS Journals), Software papers and citation in the AAS Journals
Leslie J. Sage (Nature), Software policies and guidelines at Nature
Ramon Khanna (Springer), SpringerNature data and software policies for astrophysics journals
Arfon M. Smith (STScI/JOSS), Journal of Open Source Software (JOSS): Design and first-year review
Karen Collins (Center for Astrophysics), Lessons Learned through the Development and Publication of AstroImageJ
August Muench (AAS Journals), The roles of the AAS Journals’ Data Editors
Alberto Accomazzi (NASA Astrophysics Data System), The role of the ADS in software discovery and citation
Alice Allen (ASCL/UMD), The Astrophysics Source Code Library: Supporting software publication and citation

ASCL at AAS 231

It’s AAS meeting time, and that means a busy busy week! There are some of the ASCL’s activities at this meeting; we hope to see you at our posters and in the Special Session!


Poster 150.10: The Astrophysics Source Code Library by the numbers
Tuesday, January 09, Prince Georges Exhibit Hall

Abstract: The Astrophysics Source Code Library (ASCL, ascl.net) was founded in 1999 by Robert Nemiroff and John Wallin. ASCL editors seek both new and old peer-reviewed papers that describe methods or experiments that involve the development or use of source code, and add entries for the found codes to the library. Software authors can submit their codes to the ASCL as well. This ensures a comprehensive listing covering a significant number of the astrophysics source codes used in peer-reviewed studies. The ASCL is indexed by both NASA’s Astrophysics Data System (ADS) and Web of Science, making software used in research more discoverable. This presentation covers the growth in the ASCL’s number of entries, the number of citations to its entries, and in which journals those citations appear. It also discusses what changes have been made to the ASCL recently, and what its plans are for the future.


Poster 150.28: Schroedinger’s code: Source code availability and transparency in astrophysics
Tuesday, January 09, Prince Georges Exhibit Hall

Abstract: Astronomers use software for their research, but how many of the codes they use are available as source code? We examined a sample of 166 papers from 2015 for clearly identified software use, then searched for source code for the software packages mentioned in these research papers. We categorized the software to indicate whether source code is available for download and whether there are restrictions to accessing it, and if source code was not available, whether some other form of the software, such as a binary, was. Over 40% of the source code for the software used in our sample was not available for download.

As URLs have often been used as proxy citations for software, we also extracted URLs from one journal’s 2015 research articles, removed those from certain long-term, reliable domains, and tested the remainder to determine what percentage of these URLs were still accessible in September and October, 2017.


Special Session: Astronomy Software Publishing: Community Roles and Services
Thursday, January 11, 10:00 AM – 11:30 AM; National Harbor 2
Organizers: Astrophysics Source Code Library (ASCL)/Astronomical Data Group at the Flatiron Institute

The importance of software to astronomy research is well-established, and excellent arguments to reveal these computational methods to support the research record have been advanced and much discussed in recent years. But what avenues are open to software authors to publish their codes, and what roles and services exist in the community to support their efforts? This session, organized by the Astrophysics Source Code Library (ASCL) and Astronomical Data Group at the Flatiron Institute, answers that question. It builds on previous AAS special sessions and brings together a panel of experts to present on the various avenues for publishing codes and the pros and cons of these avenues, the roles of authors, data editors, and publication indexers in software publication, the benefits of publication to authors and the discipline, and efforts of related community projects to improve aspects of software publication. After the presentations, the floor will be open for discussion and questions.

The topics and panelists are:

Introductory remarks, Peter Teuben (University of Maryland)
The evolution of software publication in astronomy, Matteo Cantiello (Flatiron Institute, Center for Computational Astrophysics/Princeton University)
Software papers and citation in the AAS journals, Chris Lintott (AAS Journals/University of Oxford)
Software policies and guidelines at Nature, Leslie J. Sage (Physical Sciences, Nature)
SpringerNature data and software policies for astrophysics journals, Ramon Khanna (Astronomy, Springer)
Journal of Open Source Software (JOSS): design and first-year review, Arfon M. Smith (Space Telescope Science Institute)
Lessons learned through the development and publication of AstroImageJ, Karen Collins (Harvard-Smithsonian Center for Astrophysics)
The roles of the AAS Journals’ Data Editors, August Muench (Journals Division, AAS)
The role of the ADS in software discovery and citation, Alberto Accomazzi (Smithsonian Astrophysical Observatory)
The Astrophysics Source Code Library: Supporting software publication and citation, Alice Allen (ASCL/University of Maryland)
Open discussion and Q&A, Moderated by Peter Teuben (University of Maryland)
Summary and closing remarks, Robert J. Nemiroff (Michigan Technological University)