Difference between revisions of "User:Shawndouglas/sandbox/sublevel13"

From LIMSWiki
Jump to navigationJump to search
(Saving and adding more.)
(91 intermediate revisions by the same user not shown)
Line 1: Line 1:
{{Infobox journal article
<div class="nonumtoc">__TOC__</div>
|name        =
{{ombox
|image        =
| type     = notice
|alt          = <!-- Alternative text for images -->
| style    = width: 960px;
|caption     =  
| text     = This is sublevel13 of my sandbox, where I play with features and test MediaWiki code. If you wish to leave a comment for me, please see [[User_talk:Shawndouglas|my discussion page]] instead.<p></p>
|title_full  = DataCare: Big data analytics solution for intelligent healthcare management
|journal     = ''International Journal of Interactive Multimedia and Artificial Intelligence''
|authors      = Baldominos, Alejandro; de Rada, Fernando; Saez, Yago
|affiliations =  Universidad Carlos III de Madrid, Camilo José Cela University
|contact      = Email: abaldomi at inf dot uc3m dot es
|editors      =
|pub_year    = 2018
|vol_iss      = '''4'''(7)
|pages        = 13–20
|doi          = [http://10.9781/ijimai.2017.03.002 10.9781/ijimai.2017.03.002]
|issn        = 1989-1660
|license      = [https://creativecommons.org/licenses/by/3.0/ Creative Commons Attribution 3.0 Unported]
|website      = [http://www.ijimai.org/journal/node/1621 http://www.ijimai.org/journal/node/1621]
|download    = [http://www.ijimai.org/journal/sites/default/files/files/2017/03/ijimai_4_7_2_pdf_16566.pdf http://www.ijimai.org/journal/sites/default/files/files/2017/03/ijimai_4_7_2_pdf_16566.pdf] (PDF)
}}
}}
{{ombox
| type      = content
| style    = width: 500px;
| text      = This article should not be considered complete until this message box has been removed. This is a work in progress.
}}
==Abstract==
This paper presents DataCare, a solution for intelligent healthcare management. This product is able not only to retrieve and aggregate data from different key performance indicators in healthcare centers, but also to estimate future values for these key performance indicators and, as a result, fire early alerts when undesirable values are about to occur or provide recommendations to improve the quality of service. DataCare’s core processes are built over a free and open-source cross-platform document-oriented database (MongoDB), and Apache Spark, an open-source cluster computing framework. This architecture ensures high scalability capable of processing very high data volumes coming at rapid speeds from a large set of sources. This article describes the architecture designed for this project and the results obtained after conducting a pilot in a healthcare center. Useful conclusions have been drawn regarding how key performance indicators change based on different situations, and how they affect patients’ satisfaction.


'''Keywords''': Architecture, artificial intelligence, big data, healthcare, management
==Sandbox begins below==
<div class="nonumtoc">__TOC__</div>
[[File:|right|400px]]
'''Title''': ''What are the key elements of a LIMS for animal feed testing?''


==Introduction==
'''Author for citation''': Shawn E. Douglas
When managing a healthcare center, there are many key performance indicators (KPIs) that can be measured, such as the number of events, the waiting time, the number of planned tours, etc. Often, keeping these KPIs within the expected limits is vital to achieving high user satisfaction.


In this paper we present DataCare, a solution for intelligent healthcare management. DataCare provides a complete architecture to retrieve data from sensors installed in the healthcare center, process and analyze it, and finally obtain relevant information, which is displayed in a user-friendly dashboard.
'''License for content''': [https://creativecommons.org/licenses/by-sa/4.0/ Creative Commons Attribution-ShareAlike 4.0 International]


The advantages of DataCare are twofold: first, it is intelligent. Besides retrieving and aggregating data, the system is able to predict future behavior based on past events. This means that the system can fire early alerts when a KPI is expected to have a future value that falls outside the expected boundaries, and it can provide recommendations for improving the behavior and the metrics, or prevent future problems with attending events.
'''Publication date''': May 2024


Second, the core system module is built on top of a big data platform. Processing and analysis are run over Apache Spark, and data are stored in MongoDB, thus enabling a highly scalable system that can process large volumes of data coming in at very high speeds.
==Introduction==
 
This article will discuss many aspects of DataCare. The next section will present context for this research by analyzing the state of the art and related work. After that an overview of DataCare’s architecture will be presented, including the three main modules responsible for retrieving data, processing and analyzing it, and displaying the resulting valuable information.


After the architecture has been explained, the subsequent three sections will describe the preprocessing, processing, and analytics engines in further detail. The design of these systems is crucial to providing a scalable solution with an intelligent behavior. After discussing those engines in detail, the article will then describe the visual analytics engine and the different dashboards that are presented to users.


Finally, the penultimate section will describe how the solution has been validated, and the last section will provide some conclusive remarks, along with potential future work.
This brief topical article will examine ...


==State of the art==
'''Note''': Any citation leading to a software vendor's site is not to be considered a recommendation for that vendor. The citation should however still stand as a representational example of what vendors are implementing in their systems.
Because healthcare services are very complex and life-critical, many works have tackled the design of healthcare management systems, aimed at monitoring metrics in order to detect undesirable behaviors that decrease their satisfaction or even threaten their safety.


Discussion on the design and implementation of the healthcare management system is not new. In the 2000s, Curtright ''et al.''<ref name="CurtrightStat00">{{cite journal |title=Strategic performance management: Development of a performance measurement system at the Mayo Clinic |journal=Journal of Healthcare Management |author=Curtwright, J.W.; Stolp-Smith, S.C.; Edell, E.S. |volume=45 |issue=1 |pages=58–68 |year=2000 |pmid=11066953}}</ref> described a system to monitor KPIs, summarizing them in a dashboard report, with a real-world application in the Mayo Clinic. Also, Griffith and King<ref name="GriffithChampion00">{{cite journal |title=Championship management for healthcare organizations |journal=Journal of Healthcare Management |author=Griffith, J.R. |volume=45 |issue=1 |pages=17–30 |year=2000 |pmid=11066948}}</ref> proposed to establish a “championship” where those healthcare systems with consistently good metrics would help improve decision making processes.
==Feed testing laboratory workflow, workload, and information management==
A feed testing lab can operate within a number of different production, research and development (R&D&#59; academic and industry), and public health contexts. They can<ref name="WardObtain24">{{cite web |url=https://animal.ifas.ufl.edu/media/animalifasufledu/dairy-website/ruminant-nutrition-symposium/archives/12.-WardRNS2024.pdf |format=PDF |author=Ward, R. |title=Obtaining value from a feed/forage lab engagement |work=Florida Ruminant Nutrition Symposium |date=27 February 2024 |accessdate=22 May 2024}}</ref>:


Some of these works explore the sensing technology that enable proposals. For instance, Ngai ''et al.''<ref name="NgaiDesign09">{{cite journal |title=Design of an RFID-based Healthcare Management System using an Information System Design Theory |journal=Information Systems Frontiers |author=Ngai. E.W.T.; Poon, J.K.L.; Suk, F.F.C.; Ng, C.C. |volume=11 |issue=4 |pages=405–417 |year=2009 |doi=10.1007/s10796-009-9154-3}}</ref> focus on how RFID technology can be applied for building a healthcare management system, yet it is only implemented in a quasi real-world setting. Ting ''et al.''<ref name="TingCritical11">{{cite journal |title=Critical elements and lessons learnt from the implementation of an RFID-enabled healthcare management system in a medical organization |journal=Journal of Medical Systems |author=Ting, S.L.; Kwok, S.K.; Tsang, A.H.; Lee, W.B. |volume=35 |issue=4 |pages=657–69 |year=2011 |doi=10.1007/s10916-009-9403-5}}</ref> also focus on the application of RFID technology to such a project, from the perspective of its preparation, implementation, and maintenance.
*act as a third-party consultant, interpreting analytical data;
*provide research and development support for new and revised formulations;
*provide analytical support for nutrition and contaminant determinations;
*provide development support for analytical methods;
*ensure quality to specifications, accreditor standards, and regulations;
*develop informative databases and data libraries for researchers;
*manage in-house and remote sample collection, labeling, and registration, including on farms; and
*report accurate and timely results to stakeholders, including those responsible for monitoring public health.


Some previous works have also tackled the design of intelligent healthcare management systems. Recently Jalal ''et al.''<ref name="JalalADepth17">{{cite journal |title=A Depth Video-based Human Detection and Activity Recognition using Multi-features and Embedded Hidden Markov Models for Health Care Monitoring Systems |journal=International Journal of Interactive Multimedia and Artificial Intelligence |author=Jalal, A.; Kamal, S.; Kim, D. |volume=4 |issue=4 |pages=54–62 |year=2017 |doi=10.9781/ijimai.2017.447}}</ref> have proposed an intelligent, depth video-based human activity recognition system to track elderly patients that could be used as part of a healthcare management and monitoring system. However, the paper does not explore this integration. Also, Ghamdi ''et al.''<ref name="GhamdiAnOnt16">{{cite journal |title=An ontology-based system to predict hospital readmission within 30 days |journal=International Journal of Healthcare Management |author=Ghamdi, H.A.; Alshammari, R.; Razzak, M.I. |volume=9 |issue=4 |pages=236–244 |year=2016 |doi=10.1080/20479700.2016.1139768}}</ref> have proposed an ontology-based system for prediction of patients’ readmission within 30 days so that those readmissions can be prevented.
This wide variety of roles further highlights the already obvious cross-disciplinary nature of analyzing animal feed ingredients and products, and interpreting the resulting data. The human [[Biology|biological]] sciences, [[Veterinary medicine|veterinary sciences]], [[environmental science]]s, [[chemistry]], [[microbiology]], [[radiochemistry]], [[botany]], [[epidemiology]], and more may be involved within a given animal feed analysis laboratory.<ref>{{Cite journal |last=Schnepf |first=Anne |last2=Hille |first2=Katja |last3=van Mark |first3=Gesine |last4=Winkelmann |first4=Tristan |last5=Remm |first5=Karen |last6=Kunze |first6=Katrin |last7=Velleuer |first7=Reinhard |last8=Kreienbrock |first8=Lothar |date=2024-02-06 |title=Basis for a One Health Approach—Inventory of Routine Data Collections on Zoonotic Diseases in Lower Saxony, Germany |url=https://www.mdpi.com/2813-0227/4/1/7 |journal=Zoonotic Diseases |language=en |volume=4 |issue=1 |pages=57–73 |doi=10.3390/zoonoticdis4010007 |issn=2813-0227}}</ref><ref name="PFPLSWHumanAnim18">{{cite web |url=https://www.aphl.org/programs/food_safety/APHL%20Documents/LBPM_Dec2018.pdf |format=PDF |title=Human and Animal Food Testing Laboratories Best Practices Manual |author=Partnership for Food Protection Laboratory Science Workgroup |date=December 2018 |accessdate=22 May 2024}}</ref><ref name=":0">{{Cite journal |last=Wood |first=Hannah |last2=O'Connor |first2=Annette |last3=Sargeant |first3=Jan |last4=Glanville |first4=Julie |date=2018-12 |title=Information retrieval for systematic reviews in food and feed topics: A narrative review |url=https://onlinelibrary.wiley.com/doi/10.1002/jrsm.1289 |journal=Research Synthesis Methods |language=en |volume=9 |issue=4 |pages=527–539 |doi=10.1002/jrsm.1289 |issn=1759-2879}}</ref> Given this significant cross-disciplinarity, it's arguably more challenging for software developers creating [[laboratory informatics]] solutions like a [[laboratory information management system]] (LIMS) that has the breadth to cover the production, R&D, and public health contexts of animal feed testing. In fact, an industry lab performing [[quality control]] (QC) work for a company will likely have zero interest in public health reporting functionality, and a LIMS that focuses on QC workflows may be more highly desirable.  


Regarding the impact of data in a healthcare management system, the importance of data-driven approaches has been addressed by Bossen ''et al.''.<ref name="BossenChallenges16">{{cite journal |title=Challenges of Data-driven Healthcare Management: New Skills and Work |journal=19th ACM Conference on Computer-Supported Cooperative Work and Social Computing |author=Bossen, C.; Danholt, P.; Ubbesen, M.B. et al. |pages=5 |year=2016 |url=http://pure.au.dk/portal/da/publications/challenges-of-datadriven-healthcare-management-new-skills-and-work(fd56833b-db7b-44ed-b4fd-15882b382271).html}}</ref> Roberts ''et al.''<ref name="RobertsADesign16">{{cite journal |title=A design thinking framework for healthcare management and innovation |journal=Healthcare |author=Roberts, J.P.; Fisher, T.R.; Trowbridge, M.J.; Bent, C. |volume=4 |issue=1 |pages=11–14 |year=2016 |doi=10.1016/j.hjdsi.2015.12.002 |pmid=27001093}}</ref> have explored how to design healthcare management systems using a design thinking framework. Basole ''et al.''<ref name="BasoleHealthcare13">{{cite journal |title=Healthcare management through organizational simulation |journal=Decision Support Systems |author=Basole, R.C.; Bodner, D.A.; Rouse, W.B. |volume=55 |issue=2 |pages=552–563 |year=2013 |doi=10.1016/j.dss.2012.10.012}}</ref> propose a web-based game using organizational simulation for healthcare management. Zeng ''et al.''<ref name="ZengVIKOR13">{{cite journal |title=VIKOR method with enhanced accuracy for multiple criteria decision making in healthcare management |journal=Journal of Medical Systems |author=Zeng, Q.L.; Li, D.D.; Yang, Y.B. |volume=37 |issue=2 |pages=9908 |year=2013 |doi=10.1007/s10916-012-9908-1 |pmid=23377778}}</ref> have proposed an enhanced VIKOR method that can be used as a decision support tool in healthcare management contexts. A relevant work from Mohapatra<ref name="MohapatraUsing15">{{cite journal |title=Using integrated information system for patient benefits: A case study in India |journal=International Journal of Healthcare Management |author=Mohapatra, S. |volume=8 |issue=4 |pages=262–71 |year=2015 |doi=10.1179/2047971915Y.0000000007}}</ref> explores how a [[hospital information system]] is used for healthcare management, improving the KPIs; and a pilot has been conducted in Kalinga hospital (India), turning out to be beneficial for all stakeholders.
That said, this Q&A article will examine LIMS functionality that addresses the needs of all three contexts for animal feed analyses. Understand that the LIMS solution your feed lab may be looking for doesn't require some of the functionality addressed here, particularly in the specialty LIMS requirements section. But also understand the broader context of feed testing and how it highlights some of the challenges of finding a feed testing LIMS that is just right for your lab.


Some works have also explored how to increase patients’ satisfaction. For example, Fortenberry and McGoldrick<ref name="FortenberryInternal15">{{cite journal |title=Internal marketing: A pathway for healthcare facilities to improve the patient experience |journal=International Journal of Healthcare Management |author=Fortenberry Jr., J.L. |volume=9 |issue=1 |pages=28–33 |year=2015 |doi=10.1179/2047971915Y.0000000014}}</ref> suggest improving the patient experience via internal marketing efforts, while Minniti ''et al.''<ref name="MinnitiPatient16">{{cite book |chapter=Patient-Interactive Healthcare Management, a Model for Achieving Patient Experience Excellence |title=Healthcare Information Management Systems |author=Minniti, M.J.; Blue, T.R.; Freed, D.; Ballen, S. |publisher=Springer |pages=257–281 |year=2016 |isbn=9783319207650 |doi=10.1007/978-3-319-20765-0_16}}</ref> propose a model in which patient feedback is processed in real time, driving rapid cycle improvement.
==Base LIMS requirements for animal feed testing==
Given the above ...


To place this work into its context, what we have developed is a data-driven intelligent healthcare management system. Because of the volume and velocity of big data, we have used a big data architecture based on the one proposed by Baldominos ''et al.''<ref name="BaldominosAScal14">{{cite journal |title=A scalable machine learning online service for big data real-time analysis |journal=2014 IEEE Symposium on Computational Intelligence in Big Data |author=Baldominos, A.; Albacete, E.; Saez, Y.; Isasi, P. |year=2014 |doi=10.1109/CIBD.2014.7011537}}</ref>, but updating the tools to use Apache Spark for the sake of efficiency. Also, a pilot has been conducted to evaluate the performance of the proposed system.
What follows is a list of system functionality important to most any feed testing laboratory, with a majority of that functionality found in many vendor software solutions.<ref name="WardObtain24" /><ref name="PFPLSWHumanAnim18" />


==Overview of the architecture==
'''Test, sample and result management'''
DataCare’s architecture comprises three main modules: the first oversees retrieving and aggregating the information generated in the health center or [[hospital]], the second processes and analyzes the data, and the third displays the valuable information in a dashboard, allowing the [[System integration|integration]] with external information systems.


Figure 1 depicts a broad overview of this architecture, while the following describes each of the modules in further detail.
*Sample log-in and management, with support for unique IDs
*Sample batching
*[[Barcode]] and RFID support
*End-to-end sample and inventory tracking, through to reporting and disposition
*Pre-defined and configurable industry-specific test and method management, including for bacteria (i.e., microbiology), heavy metals (i.e., chemistry), radionuclides (i.e., radiochemistry), and other substances
*Pre-defined and configurable industry-specific workflows, including for production, R&D, and public health contexts
*Configurable screens and data fields
*Specification management
*Test, sampling, instrument, etc. scheduling and assignment
*Test requesting
*Data import and export
*Raw data management
*Robust query tools
*Analytical tools, including [[data visualization]], statistical analysis, and [[data mining]] tools
*Document and image management
*Version control
*Project and experiment management
*Method and protocol management
*Investigation management
*Facility and sampling site management
*Storage management and monitoring


[[File:Fig1 BaldominosIntJOfIMAI2018 4-7.png|800px]]
'''Quality, security, and compliance'''
{{clear}}
{|
| STYLE="vertical-align:top;"|
{| border="0" cellpadding="5" cellspacing="0" width="800px"
|-
  | style="background-color:white; padding-left:10px; padding-right:10px;"| <blockquote>'''Figure 1.''' DataCare’s architecture. The first column lists the data sources, which are retrieved and aggregated by AdvantCare software (second column). The last column shows the big data platform, which contains engines for the data processing and analytics module (yellow) and the data visualization module (purple).</blockquote>
|-
|}
|}


===Data retrieval and aggregation module===
*[[Quality assurance]] / [[quality control]] mechanisms
Data retrieval is carried out by AdvantCare software, developed by Itas Solutions S.L. AdvantCare is a set of hardware and software tools designed to manage communications between patients and healthcare staff. Its core comprises three main systems: 1) Buslogic manages and aggregates the information of actions carried out by nondoctor personnel (nurses and nursing assistants), 2) AdvantControl monitors and controls the infrastructure, and 3) EasyConf manages voice communication.
*Mechanisms for compliance with ISO 17025 and HACCP, including support for critical control point (CCP) specifications and limits
*Result, method, protocol, batch, and material validation, review, and release
*Data validation
*Trend and control charting for statistical analysis and measurement of uncertainty
*User qualification, performance, and training management
*[[Audit trail]]s and [[chain of custody]] support
*Configurable and granular role-based security
*Configurable system access and use (i.e., authentication requirements, account usage rules, account locking, etc.)
*[[Electronic signature]] support
*Data [[encryption]] and secure communication protocols
*Archiving and [[Data retention|retention]] of data and information
*Configurable data [[backup]]s
*Status updates and alerts
*Environmental monitoring support
*Incident and non-conformance notification, tracking, and management


In hospital rooms, different data acquisition systems are placed, which often consist of hardware devices connected to an IP network and include one of the following elements:
'''Operations management and reporting'''


* sensors such as thermometers or noise or light sensors measuring some current value or status either in a continuous or periodic fashion and sending it to Buslogic or AdvantControl servers;
*Configurable dashboards for monitoring, by product, process, facility, etc.
* assistance devices such as buttons or pull handlers that are actioned by the patients and transmit the assistance call to the Buslogic server;
*Customizable rich-text reporting, with multiple supported output formats
* voice and video communication systems that send and receive information from other devices or from Jitsi (SIP Communicator), which are handled by EasyConf; or
*Custom and industry-specific reporting, including certificates of analysis (CoAs)
* data acquisition systems operated by means of graphical user interfaces in devices such as tablets, e.g., surveys or other information systems.
*Industry-compliant labeling
*Email integration and other communication support for internal and external stakeholders
*Instrument interfacing and data management, particularly for [[near-infrared spectroscopy]] (NIRS) instruments
*Third-party software interfacing (e.g., LES, scientific data management system [SDMS], other databases)
*Data import, export, and archiving
*Instrument and equipment management, including calibration and maintenance tracking
*Inventory and material management
*Supplier/vendor/customer management
*Flexible but secure client portal for pre-registering samples, printing labels, and viewing results
*Integrated (or online) system help


In general terms, the information retrieved by AdvantCare belongs to one of the following:
==Specialty LIMS requirements==


* Planned tours: Healthcare personnel will periodically visit certain rooms or patients as a part of a pre-established plan. Data about how shifts are carried out is essential to evaluate assistance quality and the efficiency of nurses and nursing assistants.
*'''Mechanisms to make data and information more FAIR''': Like many other disciplines, modern academic and industrial research of feed ingredient selection, feed formulation, and feed production is plagued by interdisciplinary research data and information (i.e., objects) "in a broad range of [heterogeneous] information formats [that] involve inconsistent vocabulary and difficult‐to‐define concepts."<ref name=":0" /> This makes increasingly attractive data discovery options<ref name=":0" /> such as text mining, cluster searching, and [[artificial intelligence]] (AI) methods less effective, in turn hampering innovation, discovery, and improved health outcomes. As such, research labs of all sorts are increasingly turning to the FAIR principles, which encourage processes that make research objects more findable, accessible, interoperable, and reusable. A handful of software developers have become more attuned to this demand and have developed or modified their systems to produce research objects that are produced using [[metadata]]- and [[Semantics|semantic-driven]] technologies and frameworks.<ref name="DouglasWhyAre24">{{cite web |url=https://www.limswiki.org/index.php/LIMS_Q%26A:Why_are_the_FAIR_data_principles_increasingly_important_to_research_laboratories_and_their_software%3F |title=LIMS Q&A:Why are the FAIR data principles increasingly important to research laboratories and their software? |author=Douglas, S.E. |work=LIMSwiki |date=May 2024 |accessdate=22 May 2024}}</ref> Producing FAIR data is more important to the academic research and public health contexts of feed testing, but can still be useful to other industrial contexts, as having interoperable and reusable data in industry can lead to greater innovation and process improvement.<ref>{{Cite journal |last=van Vlijmen |first=Herman |last2=Mons |first2=Albert |last3=Waalkens |first3=Arne |last4=Franke |first4=Wouter |last5=Baak |first5=Arie |last6=Ruiter |first6=Gerbrand |last7=Kirkpatrick |first7=Christine |last8=da Silva Santos |first8=Luiz Olavo Bonino |last9=Meerman |first9=Bert |last10=Jellema |first10=Renger |last11=Arts |first11=Derk |date=2020-01 |title=The Need of Industry to Go FAIR |url=https://direct.mit.edu/dint/article/2/1-2/276-284/10011 |journal=Data Intelligence |language=en |volume=2 |issue=1-2 |pages=276–284 |doi=10.1162/dint_a_00050 |issn=2641-435X}}</ref> Of course, all animal feed testing labs can benefit when, for example, FAIR-driven, internationally accepted vocabulary and data descriptors for mycotoxin contamination data are used in research and laboratory software.<ref>{{Cite journal |last=Mesfin |first=Addisalem |last2=Lachat |first2=Carl |last3=Vidal |first3=Arnau |last4=Croubels |first4=Siska |last5=Haesaert |first5=Geert |last6=Ndemera |first6=Melody |last7=Okoth |first7=Sheila |last8=Belachew |first8=Tefera |last9=Boevre |first9=Marthe De |last10=De Saeger |first10=Sarah |last11=Matumba |first11=Limbikani |date=2022-02 |title=Essential descriptors for mycotoxin contamination data in food and feed |url=https://linkinghub.elsevier.com/retrieve/pii/S0963996921007833 |journal=Food Research International |language=en |volume=152 |pages=110883 |doi=10.1016/j.foodres.2021.110883}}</ref> This leads into...
* Assistance tasks: Nurses and nursing assistants must perform certain tasks as a response to an assistance call. It would be great to know in advance these tasks, so they can be monitored properly.
*'''Support for standardized and controlled vocabularies''': By extension, this gets into the matter of improved interoperability of feed testing results from different laboratories, particularly government labs in different jurisdictions responsible for monitoring contaminates in animal feed.<ref name="AAFCOSACStrat22">{{cite web |url=https://www.aafco.org/wp-content/uploads/2023/07/SAC_Strategic_Plan_2023-2025.pdf |format=PDF |title=Strategic Plan 2023-2025 |author=The Association of American Feed Control Officials, Strategic Affairs Committee |publisher=AAFCO |page=14 |date=16 November 2022 |accessdate=22 May 2024}}</ref> The Association of American Feed Control Officials (AAFCO) Strategic Affairs Committee (SAC) highlight this in their Strategic Plan for 2023–2025, stating that in order to "promote and integrate laboratory technology, methods, quality systems, and collaboration in support of animal food safety systems," the different LIMS used across various states demand an integrated IT environment where "comparable results from different labs" can effectively be made.<ref name="AAFCOSACStrat22" />
* Patient satisfaction: The most important service quality subjective metric is the patient's satisfaction, which is obtained by mean of surveys.
 
As said before, AdvantCare software comprises three systems, as well as communication/integration interfaces.
 
====Buslogic====
This software oversees communication with the assistance call systems. It also handles GestCare and MediaCare, which are the systems used for tasks planning, personnel work schedules, patient information, satisfaction surveys, and entertainment. Buslogic retrieves core business information about the assistance process, including alerts, waiting times to assist patients, and achieved assistance objectives.
 
====AdvantControl====
This software controls and monitors the infrastructure and automation functionalities, including the status of lights, doors, or the DataCare infrastructure itself. It provides real-time alerts about possible quality of service issues.
 
====EasyConf====
This software manages SIP Communicator and provides data about calls such as the origin, the destination, and the total call duration.
 
====Communication/Integration APIs====
Data can be retrieved from AdvantCare servers by means of SOAP web services, which get used in those requests that require high processing capacity, and are stateless. Also, the information can be accessed via a REST [[application programming interface]] (API), where the calls are performed through HTTP requests, and data is exchanged in JSON-serialized format. REST servers are placed in the software servers themselves (either Buslogic, AdvantControl or EasyConf), thus allowing real-time queries, as well as parameter modifications. Finally, a TELNET channel will allow asynchronous communication to broadcast events from the servers to the connected clients.
 
===Data processing and analysis module===
The Data Processing and Analysis module is part of a big data platform based on Apache Spark<ref name="ZahariaSpark10">{{cite journal |title=Spark: Cluster computing with working sets |journal=Proceedings of the 2nd USENIX Conference on Hot Topics in Cloud Computing |author=Zaharia, M.; Chowdhury, M.; Franklin, M.J. et al. |page=10 |year=2010}}</ref>, which allows an integrated environment for the development and exploitation of real time massive data analysis, outperforming other solutions such as Hadoop MapReduce or Storm, scaling out up to 10,000 nodes, providing fault tolerance<ref name="ZahariaResilient12">{{cite journal |title=Resilient distributed datasets: A fault-tolerant abstraction for in-memory cluster computing |journal=Proceedings of the 9th USENIX Conference on Networked Systems Design and Implementation |author=Zaharia, M.; Chowdhury, M.; Das, T. et al. |page=2 |year=2012}}</ref> and allowing queries using a [[SQL]]-like language.
 
As shown in Figure 1, this module comprises four different systems: Preprocessing Engine, Processing Engine, Big Data and Historic Data Warehouses, and Analytics Engine.
 
====Preprocessing Engine====
This system performs the ETL (extract-transform-load) processes for the AdvantCare data. It first communicates with AdvantCare using the available APIs to retrieve the data, which later is transformed into a suitable format to be introduced to the Processing Engine. Because of the metadata provided by AdvantCare, the information can be classified to ease its analysis. Normalized and consolidated data gets stored in MongoDB, the leading free and open-source document-oriented database, where collections store both data for real time analysis as well as historic data to support batch analysis to compute the evolution of different metrics in time.
 
====Processing Engine====
This system runs over the Spark computing cluster and oversees data consolidation processes for periodically aggregating data, also supporting the alert and recommendation subsystems.
 
====Data Warehouses====
Data filtered by the Preprocessing Engine and enriched by the Processing Engine gets stored in the Big Data Warehouse, responsible for storing real-time information. Additionally, the Historic Data Warehouse stores aggregated historic data, which gets used by the Analytics Engine to identify new trends or trend shifts for the different quality metrics.
 
====Analytics Engine====
This system runs the batch processes that will apply the statistical analysis methods, as well as machine learning algorithms over real-time big data. Along with the historic data, time series and ARIMA (autoregressive integrated moving average) techniques provide diagnosis of the temporal behavior of the model. This engine also implements a Bayes-based early alerts system (EAS) able to detect and predict a decrease in the service quality or efficiency metrics under a preset threshold, sending alerts in the form of push or email notifications.
 
===Data visualization module===
This module provides a reporting dashboard that receives information from the big data platform in real time and displays two panels. The first panel shows the main quality and efficiency metrics in real time, along with its evolution over time and the quality thresholds. The second panel provides the diagnoses computed by the Analytics Engine, as well as intelligent recommendations to prevent reaching undesired situations, such as metrics falling below acceptable thresholds.
 
The dashboard is implemented using the D3.js library, providing nice and intuitive visualizations.
 
==Preprocessing Engine==
The Preprocessing Engine performs the ETL process over the data, and this section describes how different data are extracted from the various sources, transformed and loaded as a part of this process.
 
===Extraction===
This engine extracts the assistance call data by polling the AdvantCare module every five minutes, retrieving all data generated by all the rooms. Data from planned tours are retrieved daily also by polling the REST API, while patients’ satisfaction surveys are loaded as CSV files.
 
===Transformation===
The Preprocessing Engine performs several transformation tasks so that data is in a suitable format to be handled by the Processing Engine and the Analytics Engine.
 
====Assistance task events====
Assistance task events get transformed into MongoDB documents, where each event is stored in a different document, and all of them belong to the events collection. When one event
status changes (e.g., from “activated” to “notified”), the document is updated to reflect these changes.
 
Figure 2 shows a sample document representing an event.
 
<pre>{
“_id”: ObjectId(“565c234f152aee26874d7a18”),
“full_event”: true,
“presence”: {
      “ev”: “EV PRES”,
      “ts”: ISODate(“2015-10-02T01:35:36.384Z”)
},
“area”: “Madrid”,
“notification” : {
      “ev”: “EV NOTIF”,
      “ts”: ISODate(“2015-10-02T01:32:21.984Z”)
},
“room_number”: “126”,
“location”: “PERA”,
“activation” : {
      “week”: 40,
      “weekday”: 5,
      “user”: “Anonimo”,
      “hour”: 1,
      “minute”: 31,
      “year”: 2015,
      “month”: 10,
      “day”: 2,
      “ev”: “EV PERA”,
      “ts”: ISODate(“2015-10-02T01:31:45.696Z”)
},
“room_letter”: “-”,
“center”: “Aravaca”,
“day_properties”: {
      “holiday_or_sunday”: true,
      “social_events”: true,
      “rain”: true,
      “extreme_heat”: true,
      “summer_vacation”: true,
      “holiday”: true,
      “weekend”: true,
      “friday_or_eve”: true
},
“floor”: “1”,
“times”: {
      “cancellation_notification”: 195,
      “used”: 194,
      “idle”: 36,
      “cancellation_activation”: 231,
      “total”: 230,
      “cancellation_presence”: 1
},
“hour_properties”: {
      “shift_change”: true,
      “shift”: “TARDE”,
      “sleeptime”: true,
      “nurse_count”: “8”,
      “dinnertime”: true,
      “lunchtime”: true
},
“cancellation”: {
      “ev”: “EV CPRES”,
      “remote”: true,
      “ts”: ISODate(“2015-10-02T01:35:37.248Z”)
}
}</pre>
{|  
| STYLE="vertical-align:top;"|
{| border="0" cellpadding="5" cellspacing="0" width="400px"
|-
  | style="background-color:white; padding-left:10px; padding-right:10px;"| <blockquote>'''Figure 2.''' Sample JSON document representing an assistance task event in the
MongoDB events collection</blockquote>
|-  
|}
|}
 
====Planned tours====
Data from planned tours are retrieved daily from AdvantCare using the REST API and are transformed to a MongoDB document in the ''shifts'' collection. A sample document is shown in Figure 3.
 
<pre>{
      “_id”: ObjectId(“569e50b1aa40450a027eb4ec”),
      “floor”: 3,
      “room”: 326,
      “date”: “1/10/15”,
      “hour”: “9:00:45”,
      “center_name”: “Aravaca”,
      “ts”: ISODate(“2015-10-01T09:00:45.000Z”),
      “shift_type”: “MAÑANA”
}
</pre>
{|  
| STYLE="vertical-align:top;"|
{| border="0" cellpadding="5" cellspacing="0" width="400px"
|-
  | style="background-color:white; padding-left:10px; padding-right:10px;"| <blockquote>'''Figure 3.''' Sample JSON document representing a shift in the MongoDB ''shifts''
collection</blockquote>
|-
|}
|}
 
====Satisfaction surveys====
As stated before, satisfaction data are loaded as CSV files. The Preprocessing Engine transforms it into a MongoDB document, which gets stored into the surveys collection. Figure 4 shows the structure of a sample document representing a satisfaction survey.
 
<pre>
      “_id” : ObjectId(“569e483daa404509a9796754”),
      “care_punctuation”: 2,
      “center”: “Aravaca”,
      “area”: “Madrid”,
      “floor”: 2,
      “night_punctuation”: 5,
      “morning_punctuation”: 4,
      “speed_punctuation”: 2,
      “price_quality_punctuation”: 2,
      “afternoon_punctuation”: 4,
      “year”: 2015,
      “month”: 11,
      “day”: 27,
      “date”: ISODate(“2015-11-27T00:00:00.000Z”),
      “global_punctuation”: 2,
      “id”: “Anonimo”,
      “room”: 221
}
</pre>
{|  
| STYLE="vertical-align:top;"|
{| border="0" cellpadding="5" cellspacing="0" width="400px"
|-
  | style="background-color:white; padding-left:10px; padding-right:10px;"| <blockquote>'''Figure 4.''' Sample JSON document representing a satisfaction survey in the
MongoDB ''surveys'' collection</blockquote>
|-
|}
|}
 
===Load===
Once data is transformed into MongoDB documents (BSON format), they are loaded into the corresponding MongoDB collection.
 
==Processing Engine==
The Processing Engine runs batch processes to consolidate data previously transformed by the Preprocessing Engine. This consolidation aggregates data to be handled by the Analytics Engine.
 
===Periodic data consolidation===
As the Processing Engine consolidates data periodically, two new collections are created, namely ''hourly'' and ''daily'', depending on the periodicity of the aggregated data. A sample document in the ''hourly'' collection is shown in Figure 5.
 
<pre>{
“_id”: ObjectId(“5665a51f0b1d4cf6f9728ae4”),
“center”: “Aravaca”,
“date”: {
      “week”: 40,
      “weekday”: 4,
      “hour”: 4,
      “ts”: ISODate(“2015-10-01T04:00:00.000Z”),
      “year”: 2015,
      “month”: 10,
      “day”: 1
},
“idle_time”: 67,
“wait_time”: {
  “floors”: {
      “1”: 0.6363636363636364,
      “2”: 29.5,
      “3”: 120,
      “4”: 0.5
  },
  “shifts”: {
      “NOCHE”: 23.72222222222222
  },
  “total”: 427,
  “types”: {
      “EV HABA”: 4,
      “EV PERA”: 359
  }
},
“used_time”: 344,
“activity”: {
  “floors”: {
      “1”: 11,
      “2”: 2,
      “3”: 3,
      “4”: 2
  },
  “shifts”: {
      “NOCHE”: 18
  },
  “total”: 18,
  “types”: {
      “EV HABA”: 17,
      “EV PERA”: 1
  }
}
}
</pre>
{|
| STYLE="vertical-align:top;"|
{| border="0" cellpadding="5" cellspacing="0" width="400px"
|-
  | style="background-color:white; padding-left:10px; padding-right:10px;"| <blockquote>'''Figure 5.''' Sample JSON document representing consolidated data in the ''hourly''
collection</blockquote>
|-
|}
|}
 
This aggregation enables fast visualization of aggregated data, and it is key for the Analytics Engine to detect strange behaviors, fire alerts, or make recommendations. Both the ''hourly'' and ''daily'' collections are indexed by timestamp to enable fast filtering on consolidated data based on temporal queries.
 
===Real-time data processing===
To support the real-time dashboard, a process takes the data from the ''hourly'' collection and computes the average value for each KPI for different time periods: last day, last week, last month, and since the beginning. This allows comparison of the current value for a KPI with the average of past periods of time. A small fragment of a sample document in the ''realtime'' collection showing the aggregated data for the “activity” (number of events) KPI is shown in Figure 6.
 
<pre>{
      “_id” : ObjectId(“56850cb00b1d4cf6f9b4f2da”),
      “center”: “Aravaca”, “activity”: {
      “total”: [
      {“type”: “yesterday”, “hour”: 0, “value”: 106},
      {“type”: “lastweek”, “hour”: 0, “value”: 58},
      {“type”: “lastmonth”, “hour”: 0, “value”: 52},
      {“type”: “alltime”, “hour”: 0, “value”: 51.1489},
      {“type”: “yesterday”, “hour”: 1, “value”: 20},
      {“type”: “lastweek”, “hour”: 1, “value”: 33.571},
      ...
}
</pre>
{|  
| STYLE="vertical-align:top;"|
{| border="0" cellpadding="5" cellspacing="0" width="400px"
|-
  | style="background-color:white; padding-left:10px; padding-right:10px;"| <blockquote>'''Figure 6.''' Sample JSON document representing a fragment of the real-time
information for the KPI “activity” in the ''realtime'' collection</blockquote>
|-
|}
|}
 
==Analytics Engine==
The Analytics Engine is responsible for performing an intelligent analysis of the data to compute daily prediction, firing alerts when an undesired condition is detected (e.g., a certain metric falls under a specified threshold) and suggesting recommendations. This section describes these processes.
 
===Prediction system===
The prediction system takes the data contained in the events collection along with contextual data (weather, holidays, or labor dates, etc.) and predicts the estimated value for each KPI for every hour in the next day. This batch process is executed daily. The predicted values are stored in a document per each KPI, in the ''predictions'' collection in MongoDB. A sample document is shown in Figure 7.
 
<pre>{
      “_id”: ObjectId(“5683f978e4b0d671e427e1db”),
      “center”: “Aravaca”,
      “name”: “wait_time.total”,
      “date”: “1/10/15”,
      “predictions”: {
      “0”: 5637,
      “1”: 28557,
      “2”: 15711,
      “3”: 4133,
      ...
}
</pre>
{|
| STYLE="vertical-align:top;"|
{| border="0" cellpadding="5" cellspacing="0" width="400px"
|-
  | style="background-color:white; padding-left:10px; padding-right:10px;"| <blockquote>'''Figure 7.''' Sample JSON document representing a fragment of the predictions for
the “wait time” KPI in the ''predictions'' collection</blockquote>
|-
|}
|}
 
The prediction algorithm analyzes behavioral patterns in the events data and applies these patterns to simulate future behavior. The algorithm proceeds as follows for each KPI:
 
Given ''N'' clusters, the algorithm computes a matrix ''M'' where each row is a cluster and each column is an hour, thus resulting in an ''Nx''24 matrix. The value in the position ''M<sub>i,j</sub>'' contains the average value of the KPI for events happening in the cluster ''i'' and in the ''j''<sup>th</sup> hour of the day:


==Conclusion==




==References==
==References==
{{Reflist|colwidth=30em}}
{{Reflist|colwidth=30em}}
 
<!---Place all category tags here-->
==Notes==
This presentation is faithful to the original, with only a few minor changes to presentation. Grammar has been updated for clarity. In some cases important information was missing from the references, and that information was added. The original article lists references alphabetically, but this version — by design — lists them in order of appearance.
 
<!--Place all category tags here-->
[[Category:LIMSwiki journal articles (added in 2018)‎]]
[[Category:LIMSwiki journal articles (all)‎]]
[[Category:LIMSwiki journal articles on big data‎‎]]
[[Category:LIMSwiki journal articles on health informatics‎‎]]

Revision as of 21:45, 22 May 2024

This is sublevel13 of my sandbox, where I play with features and test MediaWiki code. If you wish to leave a comment for me, please see my discussion page instead.

Sandbox begins below

[[File:|right|400px]] Title: What are the key elements of a LIMS for animal feed testing?

Author for citation: Shawn E. Douglas

License for content: Creative Commons Attribution-ShareAlike 4.0 International

Publication date: May 2024

Introduction

This brief topical article will examine ...

Note: Any citation leading to a software vendor's site is not to be considered a recommendation for that vendor. The citation should however still stand as a representational example of what vendors are implementing in their systems.

Feed testing laboratory workflow, workload, and information management

A feed testing lab can operate within a number of different production, research and development (R&D; academic and industry), and public health contexts. They can[1]:

  • act as a third-party consultant, interpreting analytical data;
  • provide research and development support for new and revised formulations;
  • provide analytical support for nutrition and contaminant determinations;
  • provide development support for analytical methods;
  • ensure quality to specifications, accreditor standards, and regulations;
  • develop informative databases and data libraries for researchers;
  • manage in-house and remote sample collection, labeling, and registration, including on farms; and
  • report accurate and timely results to stakeholders, including those responsible for monitoring public health.

This wide variety of roles further highlights the already obvious cross-disciplinary nature of analyzing animal feed ingredients and products, and interpreting the resulting data. The human biological sciences, veterinary sciences, environmental sciences, chemistry, microbiology, radiochemistry, botany, epidemiology, and more may be involved within a given animal feed analysis laboratory.[2][3][4] Given this significant cross-disciplinarity, it's arguably more challenging for software developers creating laboratory informatics solutions like a laboratory information management system (LIMS) that has the breadth to cover the production, R&D, and public health contexts of animal feed testing. In fact, an industry lab performing quality control (QC) work for a company will likely have zero interest in public health reporting functionality, and a LIMS that focuses on QC workflows may be more highly desirable.

That said, this Q&A article will examine LIMS functionality that addresses the needs of all three contexts for animal feed analyses. Understand that the LIMS solution your feed lab may be looking for doesn't require some of the functionality addressed here, particularly in the specialty LIMS requirements section. But also understand the broader context of feed testing and how it highlights some of the challenges of finding a feed testing LIMS that is just right for your lab.

Base LIMS requirements for animal feed testing

Given the above ...

What follows is a list of system functionality important to most any feed testing laboratory, with a majority of that functionality found in many vendor software solutions.[1][3]

Test, sample and result management

  • Sample log-in and management, with support for unique IDs
  • Sample batching
  • Barcode and RFID support
  • End-to-end sample and inventory tracking, through to reporting and disposition
  • Pre-defined and configurable industry-specific test and method management, including for bacteria (i.e., microbiology), heavy metals (i.e., chemistry), radionuclides (i.e., radiochemistry), and other substances
  • Pre-defined and configurable industry-specific workflows, including for production, R&D, and public health contexts
  • Configurable screens and data fields
  • Specification management
  • Test, sampling, instrument, etc. scheduling and assignment
  • Test requesting
  • Data import and export
  • Raw data management
  • Robust query tools
  • Analytical tools, including data visualization, statistical analysis, and data mining tools
  • Document and image management
  • Version control
  • Project and experiment management
  • Method and protocol management
  • Investigation management
  • Facility and sampling site management
  • Storage management and monitoring

Quality, security, and compliance

  • Quality assurance / quality control mechanisms
  • Mechanisms for compliance with ISO 17025 and HACCP, including support for critical control point (CCP) specifications and limits
  • Result, method, protocol, batch, and material validation, review, and release
  • Data validation
  • Trend and control charting for statistical analysis and measurement of uncertainty
  • User qualification, performance, and training management
  • Audit trails and chain of custody support
  • Configurable and granular role-based security
  • Configurable system access and use (i.e., authentication requirements, account usage rules, account locking, etc.)
  • Electronic signature support
  • Data encryption and secure communication protocols
  • Archiving and retention of data and information
  • Configurable data backups
  • Status updates and alerts
  • Environmental monitoring support
  • Incident and non-conformance notification, tracking, and management

Operations management and reporting

  • Configurable dashboards for monitoring, by product, process, facility, etc.
  • Customizable rich-text reporting, with multiple supported output formats
  • Custom and industry-specific reporting, including certificates of analysis (CoAs)
  • Industry-compliant labeling
  • Email integration and other communication support for internal and external stakeholders
  • Instrument interfacing and data management, particularly for near-infrared spectroscopy (NIRS) instruments
  • Third-party software interfacing (e.g., LES, scientific data management system [SDMS], other databases)
  • Data import, export, and archiving
  • Instrument and equipment management, including calibration and maintenance tracking
  • Inventory and material management
  • Supplier/vendor/customer management
  • Flexible but secure client portal for pre-registering samples, printing labels, and viewing results
  • Integrated (or online) system help

Specialty LIMS requirements

  • Mechanisms to make data and information more FAIR: Like many other disciplines, modern academic and industrial research of feed ingredient selection, feed formulation, and feed production is plagued by interdisciplinary research data and information (i.e., objects) "in a broad range of [heterogeneous] information formats [that] involve inconsistent vocabulary and difficult‐to‐define concepts."[4] This makes increasingly attractive data discovery options[4] such as text mining, cluster searching, and artificial intelligence (AI) methods less effective, in turn hampering innovation, discovery, and improved health outcomes. As such, research labs of all sorts are increasingly turning to the FAIR principles, which encourage processes that make research objects more findable, accessible, interoperable, and reusable. A handful of software developers have become more attuned to this demand and have developed or modified their systems to produce research objects that are produced using metadata- and semantic-driven technologies and frameworks.[5] Producing FAIR data is more important to the academic research and public health contexts of feed testing, but can still be useful to other industrial contexts, as having interoperable and reusable data in industry can lead to greater innovation and process improvement.[6] Of course, all animal feed testing labs can benefit when, for example, FAIR-driven, internationally accepted vocabulary and data descriptors for mycotoxin contamination data are used in research and laboratory software.[7] This leads into...
  • Support for standardized and controlled vocabularies: By extension, this gets into the matter of improved interoperability of feed testing results from different laboratories, particularly government labs in different jurisdictions responsible for monitoring contaminates in animal feed.[8] The Association of American Feed Control Officials (AAFCO) Strategic Affairs Committee (SAC) highlight this in their Strategic Plan for 2023–2025, stating that in order to "promote and integrate laboratory technology, methods, quality systems, and collaboration in support of animal food safety systems," the different LIMS used across various states demand an integrated IT environment where "comparable results from different labs" can effectively be made.[8]

Conclusion

References

  1. ↑ 1.0 1.1 Ward, R. (27 February 2024). "Obtaining value from a feed/forage lab engagement" (PDF). Florida Ruminant Nutrition Symposium. https://animal.ifas.ufl.edu/media/animalifasufledu/dairy-website/ruminant-nutrition-symposium/archives/12.-WardRNS2024.pdf. Retrieved 22 May 2024. 
  2. ↑ Schnepf, Anne; Hille, Katja; van Mark, Gesine; Winkelmann, Tristan; Remm, Karen; Kunze, Katrin; Velleuer, Reinhard; Kreienbrock, Lothar (6 February 2024). "Basis for a One Health Approach—Inventory of Routine Data Collections on Zoonotic Diseases in Lower Saxony, Germany" (in en). Zoonotic Diseases 4 (1): 57–73. doi:10.3390/zoonoticdis4010007. ISSN 2813-0227. https://www.mdpi.com/2813-0227/4/1/7. 
  3. ↑ 3.0 3.1 Partnership for Food Protection Laboratory Science Workgroup (December 2018). "Human and Animal Food Testing Laboratories Best Practices Manual" (PDF). https://www.aphl.org/programs/food_safety/APHL%20Documents/LBPM_Dec2018.pdf. Retrieved 22 May 2024. 
  4. ↑ 4.0 4.1 4.2 Wood, Hannah; O'Connor, Annette; Sargeant, Jan; Glanville, Julie (1 December 2018). "Information retrieval for systematic reviews in food and feed topics: A narrative review" (in en). Research Synthesis Methods 9 (4): 527–539. doi:10.1002/jrsm.1289. ISSN 1759-2879. https://onlinelibrary.wiley.com/doi/10.1002/jrsm.1289. 
  5. ↑ Douglas, S.E. (May 2024). "LIMS Q&A:Why are the FAIR data principles increasingly important to research laboratories and their software?". LIMSwiki. https://www.limswiki.org/index.php/LIMS_Q%26A:Why_are_the_FAIR_data_principles_increasingly_important_to_research_laboratories_and_their_software%3F. Retrieved 22 May 2024. 
  6. ↑ van Vlijmen, Herman; Mons, Albert; Waalkens, Arne; Franke, Wouter; Baak, Arie; Ruiter, Gerbrand; Kirkpatrick, Christine; da Silva Santos, Luiz Olavo Bonino et al. (1 January 2020). "The Need of Industry to Go FAIR" (in en). Data Intelligence 2 (1-2): 276–284. doi:10.1162/dint_a_00050. ISSN 2641-435X. https://direct.mit.edu/dint/article/2/1-2/276-284/10011. 
  7. ↑ Mesfin, Addisalem; Lachat, Carl; Vidal, Arnau; Croubels, Siska; Haesaert, Geert; Ndemera, Melody; Okoth, Sheila; Belachew, Tefera et al. (1 February 2022). "Essential descriptors for mycotoxin contamination data in food and feed" (in en). Food Research International 152: 110883. doi:10.1016/j.foodres.2021.110883. https://linkinghub.elsevier.com/retrieve/pii/S0963996921007833. 
  8. ↑ 8.0 8.1 The Association of American Feed Control Officials, Strategic Affairs Committee (16 November 2022). "Strategic Plan 2023-2025" (PDF). AAFCO. p. 14. https://www.aafco.org/wp-content/uploads/2023/07/SAC_Strategic_Plan_2023-2025.pdf. Retrieved 22 May 2024. 
Retrieved from "https://www.limswiki.org/index.php?title=User:Shawndouglas/sandbox/sublevel13&oldid=63688"