Journal:Generalized procedure for screening free software and open-source software applications/Summary and glossary

From LIMSWiki
Jump to navigationJump to search
Full article title Generalized Procedure for Screening Free Software and Open Source Software Applications
Author(s) Joyce, John
Author affiliation(s) Arcana Informatica; Scientific Computing
Primary contact Email:
Year published 2015
Distribution license Creative Commons Attribution-ShareAlike 4.0 International
Website Print-friendly version
Download PDF (Note: Inline references fail to load in PDF version)


Free software and open-source software projects have become a popular alternative tool in both scientific research and other fields. However, selecting the optimal application for use in a project can be a major task in itself, as the list of potential applications must first be identified and screened to determine promising candidates before an in-depth analysis of systems can be performed. To simplify this process, we have initiated a project to generate a library of in-depth reviews of free software and open-source software applications. Preliminary to beginning this project, a review of evaluation methods available in the literature was performed. As we found no one method that stood out, we synthesized a general procedure using a variety of available sources for screening a designated class of applications to determine which ones to evaluate in more depth. In this paper, we examine a number of currently published processes to identify their strengths and weaknesses. By selecting from these processes we synthesize a proposed screening procedure to triage available systems and identify those most promising of pursuit. To illustrate the functionality of this technique, this screening procedure is executed against a selected class of applications.



Literature review

Initial evaluation and selection recommendations

In-depth evaluation

Completing the evaluation


In support of a project to prepare published evaluations of various FLOSS applications, we have reviewed the FLOSS literature, focusing particularly on any assessment or evaluation documents. While many described proposed evaluation methods, none of them appear to be particularly popular or have developed an active community around them. Several review papers on the topic, while identifying multiple methods and their advantages, found flaws in all of them, particularly in terms of being able to perform a quality assessment on any FLOSS application. Many of the described systems were explicitly focused on a single class of FLOSS applications, such as library management systems.

By consolidating suggestions and procedures from a number of these papers, we synthesized a general survey process to allow us to quickly assess the status of any given type of FLOSS applications, allowing us to triage them and identify the most promising candidates for in-depth evaluation. Note that this process is designed for performing high-level surveys, it is not designed to perform the in-depth evaluations required for product selection.

As a minor aside, in the course of researching this article I was surprised by the high percentage of the published papers on FLOSS which were published in classic subscription journals, as opposed to any of the various open-source journals available.[1] Somehow it seems like a curious disconnect not to publish articles on open-source Software in open-source journals. Whether this is just habit of submission or due to more considered reasons would be interesting to know.


Term Explanation
.ogg File extension for Ogg Vorbis, an open-source patent-free audio compression format
21 CFR Part 11 United States Food and Drug Administration electronic records and electronic signatures rule A web site that allows academics to share research papers and exchange information
accuracy Defines how well the results of an analysis or measurement conforms to the actual or "correct" value
AEQ Analytical equipment qualification
agnostic A term applied to both hardware and software that indicates that the item is interoperable with different systems (The correct term should probably be "technology-independent" or "technology-neutral," but the use of the term "agnostic" appears to be well entrenched.)
AHP Analytic hierarchy process (AHP), a structured technique developed by Thomas Saaty in the 1970s for organizing and analyzing complex decisions
AIQ Analytical instrument qualification, a term used in the pharmaceutical industry for the process of ensuring that an instrument meets the requirements for its intended application
Apache Hadoop YARN Yet Another Resource Negotiator (YARN), a technology for managing resources on a cluster of computers
Apache Lucene A software library for information retrieval from fields of text contained within document files
ASE Adaptive Server Enterprise
audit trail A log of records documenting the sequence of activities that have been performed on a system
binary package distribution A compilation of the compiled version of a program and all related documentation designed for release to the end user
BREW Binary Runtime Environment for Wireless, a runtime and application development environment from Qualcomm that isolates portable applications from the hardware interface of mobile phones employing code division multiplex access (CDMA)
BRR Business readiness rating
CDMS Chromatography data management system
CFR United States Code of Federal Regulations
change log A log documenting the changes made to a software product, which may include a list of new features, changes to behavior, or elimination of software bugs
COC Chain of custody, a paper or electronic documentation trail that documents responsibility of a sample (This is required for legal reasons under various regulatory programs, and it also provides information used to track faulty or contaminated items back to their source.)
committer The person that represents the quality control of the community, controlling what changes are included in the originally licensed version, though users are free to make any changes they want in their own copies of the program
Compiere A suite of open-source applications (for small- to medium-sized businesses) that provides a number of business support applications
copyleft Per Wikipedia:

Copyleft (a play on the word "copyright") is the practice of offering people the right to freely distribute copies and modified versions of a work with the stipulation that the same rights be preserved in derivative works down the line.

Copyleft is a form of licensing, and can be used to maintain copyright conditions for works ranging from computer software, to documents, to art. In general, copyright law is used by an author to prohibit recipients from reproducing, adapting, or distributing copies of their work. In contrast, under copyleft, an author may give every person who receives a copy of the work permission to reproduce, adapt, or distribute it, with the accompanying requirement that any resulting copies or adaptations are also bound by the same licensing agreement.

Copyleft licenses (for software) require that information necessary for reproducing and modifying the work must be made available to recipients of the binaries. The source code files will usually contain a copy of the license terms and acknowledge the author(s).

Copyleft type licenses are a novel use of existing copyright law to ensure a work remains freely available. The GNU General Public License, originally written by Richard Stallman, was the first software copyleft license to see extensive use, and continues to dominate in that area. Creative Commons, a non-profit organization founded by Lawrence Lessig, provides a similar license provision condition called ShareAlike.

Cosmos C# Open Source Managed Operating System
COTS Commercial, off-the-shelf
CROMERR Cross-Media Electronic Reporting Rule (or typically 40 CFR Part 3), an Environmental Protection Agency (EPA) rule specifying how electronic reporting should be performed for the EPA's various regulatory programs
Cygwin A collection of tools that emulate a Linux environment, allowing Linux applications to be compiled for and executed in a MS Windows environment
data loading The loading of data into static tables, including test definitions, sample container descriptions, location information, etc.
database-agnostic Indicates an application capable of running with database systems from any vendor
digital commons Name given to a collaboratively developed online resource that is managed by a community of people
DMOZ Also known as the Open Directory Project (ODP), an attempt to create the largest human curated open-content directory of web links (Originates from one of its earlier domain names,
Docker A system designed to package an application with all of its dependencies into a standardized software container using an alternate architectural approach than a virtual machine, while providing similar resource isolation in a smaller footprint
documentation, administrator Documentation for a system designed to be issued to a system administrator, providing information on configuring and operating the application
documentation, developer Documentation for a system designed to be issued to a system developer, providing detailed information on how the system operates and is structured as well as how to alter the system code and extract data from the system in an ad hoc manner
documentation, user Documentation for a system designed to be issued to a user, helping to guide the user through using the application
DOI Digital object identifier, a character string identifier used to uniquely identify an electronic document (In many instances a scientific/technical publication will have a DOI printed on it, allowing you to access or purchase the article on-line; standardized under ISO 26324.)
Drizzle A MySQL 6.0-derived database optimized for cloud computing
Drupal An open-source application designed for creating and managing a variety of web sites
Eclipse A Java-based integrated development environment that can be customized via use of community-developed plug-ins
EDD Electronic data delivery or electronic data deliverable
EHR Electronic health record
Elasticsearch An open-source search engine employing a RESTful (see "REST") interface, built on Apache Lucene
ELN Electronic laboratory notebook
EMR Electronic medical record
EPA United States Environmental Protection Agency
ERP Enterprise resource planning, typically constituting a variety of integrated applications, with a shared database, that integrates critical business functions such as accounting, human resources, customer relationship management, inventory and order tracking, etc. into a single system
F/OSS Free/open-source software (See "FOSS")
FAME Filter, Analyze, Measure, and Evaluate (FAME) methodology for evaluating open-source applications
FDA United States Food and Drug Administration
Fedora A version of the Linux operating system sponsored by Red Hat
Firefox One of a number of open-source web browsers that can be highly customized via the use of member-developed "plug-ins"
FLOSS Per Wikipedia:

The acronym FLOSS was coined in 2001 by Rishab Aiyer Ghosh for "free/libre and open-source software." Later that year, the European Commission (EC) used the phrase when they funded a study on the topic. Unlike "libre software," which aimed to solve the ambiguity problem, FLOSS aimed to avoid taking sides in the debate over whether it was better to say "free software" or to say "open-source software"

FLOSShub Per FLOSS Research Group:

FLOSShub is a portal for free/libre and open source software (FLOSS) research resources and discussion. FLOSShub's goal is to provide a central location for connecting researchers and FLOSS community members to research papers, data, tools, and most importantly, community.

Forge Per Wikipedia:

In FLOSS development communities, a forge is a web-based collaborative software platform for both developing and sharing computer applications. (The word derives from the metalworking forge, used for shaping metal parts.) A forge platform is generally able to host multiple independent projects.

For software developers it is a place to host, among others, source code (often version-controlled), bug database and documentation for their projects. For users, a forge is a repository of computer applications.

Software forges have become popular, and have proven successful as a software development model for a large number of software projects.

The term forge refers to a common prefix or suffix adopted by various platforms created after the example of SourceForge (such as GForge and FusionForge).

forking The process where a group takes an open-source application's source code and starts developing it in an independent direction from the original program
FOSS Per Wikipedia:

Free and open-source software (FOSS) is computer software that can be classified as both free software and open-source software. That is, anyone is freely licensed to use, copy, study, and change the software in any way, and the source code is openly shared so that people are encouraged to voluntarily improve the design of the software. This is in contrast to proprietary software, where the software is under restrictive copyright and the source code is usually hidden from the users.

free software See "software, free"
Free Software Foundation Per Wikipedia:

The Free Software Foundation (FSF) is a 501(c)(3) non-profit organization founded by Richard Stallman on 4 October 1985 to support the free software movement, which promotes the universal freedom to study, distribute, create, and modify computer software, with the organization's preference for software being distributed under copyleft ("share alike") terms, such as with its own GNU General Public License. The FSF was incorporated in Massachusetts, USA, where it is also based.

From its founding until the mid-1990s, FSF's funds were mostly used to employ software developers to write free software for the GNU Project. Since the mid-1990s, the FSF's employees and volunteers have mostly worked on legal and structural issues for the free software movement and the free software community.

Consistent with its goals, only free software is used on the FSF's computers.

functional requirement Per Wikipedia:

In software engineering (and systems engineering), a functional requirement defines a function of a system and its components. A function is described as a set of inputs, the behavior, and outputs (see also software). Functional requirements may be calculations, technical details, data manipulation and processing and other specific functionality that define what a system is supposed to accomplish. Behavioral requirements describing all the cases where the system uses the functional requirements are captured in use cases. Functional requirements are supported by non-functional requirements (also known as quality requirements), which impose constraints on the design or implementation (such as performance requirements, security, or reliability). Generally, functional requirements are expressed in the form "system must do <requirement>", while non-functional requirements are "system shall be <requirement>". The plan for implementing functional requirements is detailed in the system design. The plan for implementing non-functional requirements is detailed in the system architecture.

GALP Good Automated Laboratory Practices
GCP Good Clinical Practice
GCP Good Laboratory Practice
GMP Good Manufacturing Practice
GNU GNU’s Not Unix, a project to create a FLOSS operating system
GNU General Public License Per Wikipedia:

The GNU General Public License (GNU GPL or GPL) is the most widely used free software license, which guarantees end users (individuals, organizations, companies) the freedoms to run, study, share (copy), and modify the software. Software that allows these rights is called free software and, if the software is copylefted, requires those rights to be retained. The GPL demands both. The license was originally written by Richard Stallman of the Free Software Foundation (FSF) for the GNU project. In other words, the GPL grants the recipients of a computer program the rights of the Free Software Definition and uses copyleft to ensure the freedoms are preserved whenever the work is distributed, even when the work is changed or added to. The GPL is a copyleft license, which means that derived works can only be distributed under the same license terms. This is in distinction to permissive free software licenses, of which the BSD licenses and the MIT License are the standard examples. GPL was the first copyleft license for general use.

GNU Project Per Wikipedia:

The GNU Project is a free software, mass collaboration project, announced on 27 September 1983, by Richard Stallman at MIT. Its aim is to give computer users freedom and control in their use of their computers and computing devices, by collaboratively developing and providing software that is based on the following freedom rights: users are free to run the software, share it (copy, distribute), study it and modify it. GNU software guarantees these freedom-rights legally (via its license), and is therefore free software; the use of the word "free" always being taken to refer to freedom.
In order to ensure that the entire software of a computer grants its users all freedom rights (use, share, study, modify), even the most fundamental and important part, the operating system (including all its numerous utility programs), needed to be written. The founding goal of the project was, in the words of its initial announcement, to develop "a sufficient body of free software [...] to get along without any software that is not free." Stallman decided to call this operating system GNU (a recursive acronym meaning "GNU's not Unix"), basing its design on that of Unix; however, in contrast to Unix which was proprietary software, GNU was to be freedom-respecting software (free software) that users can use, share, study and modify. Development was initiated in January 1984. The goal of making a completely free software operating system was achieved in 1992 when the third-party Linux kernel was released as free software, under version 2 of the GNU General Public License, to be used with the GNU software stack.

Google Scholar A version of Google search optimized for searching and retrieving scholarly papers
GPL See "GNU General Public License"
HDL hardware description language
HL7 A non-profit organization working to define standard methods for the exchange and retrieval of medical information
IEC International Electrotechnical Commission
illegal operation Term for an operating system command that is unknown to the operating system or processor
ISO International Organization for Standardization, an independent, non-governmental membership organization and the world's largest developer of voluntary International Standards
Jabber An XML based instant messaging platform
LIMS Laboratory information management system, an informatics system designed to track samples and analytical results through a laboratory
LIS Laboratory information system, an informatics system designed to track samples and test results through a clinical laboratory
LMS Library management system
Microsoft Academic Search An experimental Microsoft search project focusing on scientific and technical information
modular A design process where an application is broken down into smaller parts, frequently grouped by function
MongoDB An open-source NoSQL document-oriented database
native code A program that is designed to run with a specific computer's hardware program codes, on a different computer/processor via the use of an emulator
non-functional requirements Requirements describing how the system works and how it should behave
OALib Open Access Library, an open-access search engine and journal
ODBC Open Database Connectivity, a middleware API that allows an application to connect with any database
ODF Open Document Format, an XML based open-source document format developed for office suite use, prominent in the OpenOffice and LibreOffice projects
ODP Open Directory Project; see "DMOZ"
OSI Open Source Initiative, a global non-profit organization promoting open source and preventing abuse of the open source concept; also the steward of the Open Source Definition (OSD)
OpenDOAR Directory of Open Access Repositories
OpenLogic Exchange Software-as-a-Service (SaaS) governance platform for comprehensive governance and provisioning of open-source software
OpenSSL A general-purpose encryption library used in many web servers that provides an implementation of the Internet's Secure Sockets Layer (SSL) and Transport Layer Security (TLS) protocols
OpenVPN Open-source virtual private network (VPN) software; see "VPN"
OPML Outline Processor Markup Language, an XML-based file format used for creating outlines
OS Operating system
OSDL Open Source Development Labs, conceived as a facility to allow open source developers to collaborate to create standardized implementations of Linux and Linux applications by IBM, Intel, and Computer Associates
OSS Per Wikipedia:

Open-source software (OSS) is computer software with its source code made available with a license in which the copyright holder provides the rights to study, change, and distribute the software to anyone and for any purpose. Open-source software may be developed in a collaborative public manner. Open-source software is the most prominent example of open-source development and often compared to (technically defined) user-generated content or (legally defined) open-content movements.

OSS Watch An independent non-advocacy service on free and open-source software
PDA Parenteral Drug Association
PDF Adobe Portable Document Format
Perl An interpreted script programming language invented by Larry Wall that is popular for creating common gateway interface (CGI) programs
precision How closely a set of measurements agree with their average
proprietary software See "software, proprietary"
PROSE Promoting Open Source in Europe
QA Quality assurance, responsible for auditing the QC process and ensuring that all SOPs and standards are being followed
QC Quality control, responsible for ensuring all parts of a process are within designed specifications (designated testing and inspection)
QMS Quality management system
QSOS Qualification and Selection Open Source, a methodology for evaluating open-source applications backed by Atos Origin
REST Representational state transfer (occasionally ReST), an architectural style commonly used for APIs that is defined by six constraints: uniform interface, stateless, cacheable, client-server, layered system, and code on demand
ResearchGate A professional network allowing scientists and researchers to exchange papers and collaborate
revenue trigger A method of bringing in money to support program development, appearing as ad sales to training, consulting contracts, and/or support subscriptions
RFI Request for information, a document sent by potential customers to vendors requesting detailed information regarding their system, usually in relation to selecting a hardware or software system to acquire
RFID Radio frequency identification device
RFP Request for proposal, a document sent to vendors specifying application requirements and requesting a proposal of what they can supply
Ruby An open-source object-oriented scripting/programming language created by Yukihiro Matsumoto
Samba An implementation of Server Message Block (SMB) and Common Internet File System (CIFS) client/server protocols that allows shared access of resources over a network
shareware Per Wikipedia:

Shareware is a type of proprietary software which is provided (initially) free of charge to users, who are allowed and encouraged to make and share copies of the program, which helps to distribute it. The word "shareware" is a portmanteau combining the words "share" and "software". Shareware is often offered as a download from an Internet website or as a compact disc included with a magazine.

There are many types of shareware, and while they may not require an initial up-front payment, all are intended to generate revenue in one way or another. Some limit use to personal non-commercial purposes only, with purchase of a license required for use in a business enterprise. The software itself may be limited in functionality or be time-limited. Or it may remind you that payment would be appreciated.

Shareware is available on all major personal computer platforms. Titles cover a very wide range of categories including: business, software development, education, home, multimedia, design, drivers, games, and utilities. Because of its minimal overhead and low cost, the shareware model is often the only one practical for distributing non-free software for abandoned or orphaned platforms such as the Atari ST and Amiga.

The term shareware is used in contrast to open-source software, in which the source code is available for anyone to inspect and alter, and freeware, which is software distributed at no cost to the user but without source code being made available. Note that two types of shareware, donationware and freemiums, are also types of freeware.

Smalltalk An experimental language developed at Xerox in the 1970s to investigate the concept of object-oriented programming
SME Subject matter expert, a person who is an authority on the particular subject area or topic that a software application addresses
software, free Per Wikipedia:

Free software, software libre, or libre software is computer software that gives users the freedom to run the software for any purpose as well as to study, modify, and distribute the original software and the adapted versions. The rights to study and modify free software imply unfettered access to its source code. For computer programs which are covered by copyright law this is achieved with a software license where the author grants users the aforementioned freedoms. Software which is not covered by copyright law, such as software in the public domain can also be free if the source code is in the public domain (or otherwise available without restrictions). Other legal and technical aspects such as software patents and DRM may impede users from exercising these rights, and thus prevent software from being free. Free software may be developed collaboratively by volunteer computer programmers or by corporations; as part of a commercial activity or not.
Free software is primarily a matter of liberty, not price: users, individually or collectively, are free to do whatever they want with it – this includes the freedom to redistribute the software free of charge, or to sell it (or related services such as support or warranty) for profit. Free software thus differs from proprietary software (such as Microsoft Windows), which to varying degrees prevents users from studying, modifying and sharing the software. Free software is also distinct from freeware, which is simply a category of proprietary software which does not require payment for use. Proprietary software (including freeware) uses restrictive software licences or user agreements and usually does not provide access to the source code. Users are thus prevented from modifying the software, and this results in the user becoming dependent on software companies to provide updates and support (vendor lock-in). Users can also not necessarily reverse engineer, modify, or redistribute proprietary software.

software, open-source Per Wikipedia:

Open-source software (OSS) is computer software with its source code made available with a license in which the copyright holder provides the rights to study, change, and distribute the software to anyone and for any purpose. Open-source software may be developed in a collaborative public manner. Open-source software is the most prominent example of open-source development and often compared to (technically defined) user-generated content or (legally defined) open-content movements.

software, proprietary Per Wikipedia:

Proprietary software, non-free software (in the sense of missing freedoms), or closed-source software is software, where the developers or distributors reserve all freedoms and rights.

Among the freedoms and rights that proprietary software deprives (to end-users), are:

  • the freedom to analyze the software, and to change it (often deprived through intentional non-availability of sourcecode, or through Non-disclosure agreements (NDA))
  • the freedom to share the software (often deprived through copy prohibition via EULA (End User License Agreement) or NDA)
  • the freedom to run the software for any purpose (often deprived through user-restrictions via EULA).

In contrast to proprietary software, free software, is software that grants a user all these freedoms, on reception of the software.

Proprietary software is licensed under legal right of the copyright holder, with the intent that the licensee is given the right to use the software only under certain conditions, and restricted from other uses, such as modification, sharing, studying, redistribution, or reverse engineering. Usually the source code of proprietary software is not made available.

Complementary terms include free software, licensed by the owner under more permissive terms, and public domain software, which is not subject to copyright and can be used for any purpose. Proponents of free and open-source software use proprietary or non-free to describe software that is not free or open-source.

A related, but distinct categorization in the software industry is commercial software, which refers to software produced for sale, but without meaning it is closed-source.

SOP Standard operating procedure
source code A version of the program in human-readable form
source package distribution A consolidation of all of the source code and documentation relevant to a specific software release intended for the application developer and/or maintainer
SourceForge A software repository on the web providing support for open source developers and for distribution of their application
spaghetti code A contemptuous phrase applied to unstructured applications, particularly those with GOTO statements or other control structures bouncing control around different portions of the program
SQA Software quality assurance, responsible for auditing the SQC process and ensuring that all SOPs and standards are being followed
SQC Software quality control, responsible for ensuring all parts of a process were within designed specifications (designated testing and inspection)
SQL Structured Query Language
Subversion An open-source program designed by Karl Fogel and Ben Collins-Sussman to serve as a version control system that tracks changes made to files and folders
system suitability testing Applied to analytical procedures, the idea that everything involved in the analysis (analytical equipment, data capture electronics, analytical procedures, and analytical samples) constitutes a system and can be evaluated as such
tcpdump A command line tool for monitoring network traffic, also useful in troubleshooting network problems
Tsunami UDP A file transfer protocol enabling high-speed data transfers over networks with large end-to-end delays, often used for bulk data transfers
URI Per Wikipedia:

In computing, a uniform resource identifier (URI) is a string of characters used to identify the name of a resource. Such identification enables interaction with representations of the resource over a network, typically the World Wide Web, using specific protocols. Schemes specifying a concrete syntax and associated protocols define each URI. The most common form of URI is the uniform resource locator (URL), frequently referred to informally as a web address. More rarely seen in usage is the uniform resource name (URN), which was designed to complement URLs by providing a mechanism for the identification of resources in particular namespaces.

URL Universal resource locator, a specific type of uniform resource identifier that identifies objects on the World Wide Web, including their address
URS User requirements specification, a document that lists all of the requirements that the users require the system to support, frequently considered the key document in regards to the development life cycle of the application
user experience The overall aspects that an end-user experiences when interacting with the application, distinct from the functionality of the user interface
USP Per the USP:

The U.S. Pharmacopeial Convention (USP) is a scientific nonprofit organization that sets standards for the identity, strength, quality, and purity of medicines, food ingredients, and dietary supplements manufactured, distributed and consumed worldwide. USP’s drug standards are enforceable in the United States by the Food and Drug Administration, and these standards are used in more than 140 countries.

validation A process of confirming that the application works as designed
vendor-neutral A business approach designed to avoid lock-in with a particular supplier and ensure broad compatibility and interchangeability of products and technologies
versioning The assignment of a unique name or number to a given compilation of software, frequently in terms of a major and minor version number
VM Virtual machine, a program running on another computer that emulates a full computer system (Originally developed to allow more efficient use of idle time on existing hardware servers)
VPN Per Wikipedia:

A virtual private network (VPN) extends a private network across a public network, such as the Internet. It enables users to send and receive data across shared or public networks as if their computing devices were directly connected to the private network, and thus are benefiting from the functionality, security and management policies of the private network. A VPN is created by establishing a virtual point-to-point connection through the use of dedicated connections, virtual tunneling protocols, or traffic encryption.

WAN Wide area network, a computer network connecting multiple locations, as opposed to a local area network (LAN) that services one location
wiki A server program designed to promote collaborating development and used in a diverse variety of ways, including system documentation
WLAN Wireless local area network
XBRL Extensible Business Reporting Language
XML Extensible Markup Language


  1. Glover, H.D. (2013). "Academia Should Embrace Open Access Scholarly Publishing". Open Journal of Accounting 2 (4): 95–96. doi:10.4236/ojacct.2013.24012. 


This article has not officially been published in a journal. However, this presentation is largely faithful to the original paper. The content has been edited for grammar, punctuation, and spelling. Additional error correction of a few reference URLs and types as well as cleaning up of the glossary also occurred. Redundancies and references to entities that don't offer open-source software were removed from the FLOSS examples in Table 2. DOIs and other identifiers have been added to the references to make them more useful. This article is being made available for the first time under the Creative Commons Attribution-ShareAlike 4.0 International license, the same license used on this wiki.