Skip To Content

A Guide to Software Evaluation for Busy Scientists: Remember to Mind the Gap

December 3, 2020
by Andrew Anderson, Vice President of Innovation & Information, ACD/Labs



Having just marked my 5-year anniversary back with ACD/Labs, I’ve been thinking a lot about our mission and about change. As the technological ecosystem changes, work processes must also change. At ACD/Labs, helping users with this transition is our goal. We’ve learned a lot about implementing new software in companies over the last 5 years, and we’ve seen its transformative effect on users’ daily routines.

But the software-evaluation project starts before us—it begins when our customers first identify and articulate the gaps in their process. So I’d like to share my perspective on technology scouting from my time at PepsiCo. My department there looked for “Technology Unlocks” for R&D—tech that could fill gaps in their workflows. For example, this methodology helped R&D implement colorimetric sweat patches for athletes and analyze professional chefs’ work to unlock the secrets of healthy flavors.

Here’s the process I used for these projects. I hope it’ll help those of you who need to evaluate scientific software. Tech scouting can seem like a daunting project, especially when you have a million other tasks to complete, but taking it step by step will help you find technology that unlocks new opportunities and efficiencies.

The five steps to scientific-software evaluation

  1. Define the gap
    Articulate the critical points where technology would improve a process. One way to do this: identify all the actions in a process, and pick out the ones that technology would make more efficient, or eliminate altogether.
  2. Quantify the human and institutional costs of the gap
    What is the cost (in money, time, and effort) of continuing with the status quo? For example, if you frequently prepare reports that that contain multiple types of analytical data, you must gather data from multiple software systems and consolidate it. Measure the time required to do this each week or month or year. If technology can reduce or eliminate this time, you can rationalize the cost of implementing a new technology.
  3. Itemize the gap-filling requirements, both functional and non-functional
    What features are required to fill the gap? These include both functional and non-functional requirements.
    Functional requirements are tasks that you as a user would like to complete. For example, you may sometimes process data manually after it’s acquired and automatically processed by an instrument. If you don’t like how a peak is integrated, you’ll need to open the original dataset, manually adjust peak and baseline detection or integration settings, and update the summary analysis. These “to-do list” tasks are functional requirements.
    Non-functional requirements are the background capabilities that make a functional requirement possible. Think about how feature requirements would be delivered (e.g., what systems your current infrastructure can support) and your performance expectations (e.g., how fast a task needs to be performed).In the example above, you would need to efficiently query your data for chemical, spectral, and chromatographic data or metadata to find the original dataset.
    As another example, you might want all your data automatically collected in a central database. Then the instrument software systems need to be exposed to your network, so their data can be retrieved. (Some non-functional requirements will need input from other stakeholders, like IT. See Step 6 to read more about stakeholder consultation.)
  4. Prepare a requirements-to-value traceability matrix
    Link the costs and benefits identified in Step 2 to the requirements from Step 3. A requirements-to-value traceability matrix lets you score each candidate technology, then trace the score back to the actual human and institutional costs it mitigates. When you’re ready to select a technology, you can predict its value to you and your organization.Software-Evauation-mind-the-gaps2
    A value (saving 100 work-hours per year for scientists) traces to a requirement (automatic data uploading to a database), on which candidate technologies can be scored. You can read the chart the other way as well: if a technology scores highly, it fulfills the requirement, and scientists will save time on this process.
  5. Identify and consult with implementation stakeholders
    Invite stakeholders at the right time. You need to know that there are some possibilities; there’s no point in asking for people’s time only to decide the prospective technologies don’t meet your own requirements. On the other hand, if you get too far along, other teams will find it difficult to register their concerns. At a minimum, prepare lists of functional requirements and prospective technologies. Then invite others to help you rank non-functional requirements.
    Colleagues who will help you implement the technology might include IT, Facilities, Finance, Procurement, and Legal. They’ll help you figure out what’s required to put the technology in place in your environment, while keeping compliance. Also, include colleagues in other departments who’ll be affected by the technology implementation, such as downstream teams who might receive a new report or different results.

That’s the view from the technology scouter’s end as you prepare to browse and evaluate software. Of course, the vendor can help as well. My ACD/Labs colleagues in our customer-facing departments have years of experience helping prospective customers perform their scientific and functional due diligence during product evaluation. They can also help you justify the value of implementing a new technology. If you’re interested in learning more, contact us.


Your email address will not be published. Required fields are marked *