Accelerating Structure Verification Across Novartis

Open Access NMR Workflow

The AXS NMR Group manages 15 instruments across their site, eight of which are open access. Four of these instruments support their highly automated Lab2Lab workflow, which enables chemists to submit samples to instruments in other buildings—via pneumatic tubes—without leaving their lab. The remaining four open access instruments are part of a more manual walk-up workflow, providing greater flexibility and control over the process but requiring more time and effort with a higher risk of human error. Regardless of which workflow the chemists use, all 1D ¹H and ¹³C NMR data generated in open access workflows is automatically subject to automated structure verification (ASV) with NMR Workbook Suite™.

Users of the open access system are varied, which also means that there is a wide diversity of samples submitted for analysis. The samples differ based on properties such as exchange phenomena, substance concentration, and molecular size. To optimize the quality of spectra recorded—and thus the accuracy of ASV—while maintaining efficiency on the open access instruments, it is crucial to adjust NMR experiment parameters to individual sample properties. To do this, Novartis has an in-house script that automatically runs after acquisition of the 1D ¹H NMR spectrum to calculate the signal-to-noise ratio (s/n). The system then uses this result to determine the optimal parameter for subsequent ¹³C and 2D experiments. (Figure 1)

Once the data is collected and analyzed by ASV, the raw and processed data as well as a report of the ASV results are automatically emailed to the chemist. This gives them the opportunity to review and adjust any of the automated analysis or assignments.

Assessing the Accuracy of ASV

To get an idea of how well ASV is performing in this environment, we can look at a snapshot of the results from all samples run on a single day. On this day, there were 116 samples submitted covering a wide range of experiment types—from a simple 1D proton spectrum to comprehensive sets of 2D experiments. The molecules under scrutiny were also diverse with molecular weights ranging from 126–1168 Da, and containing fluorine and phosphorus heteroatoms as well as a variety of heterocycles.

All but 4% of samples had experimental data that was applicable to their ASV system. The structure was accurately verified (true positive) or rejected (true negative) for 40% and 15% of samples respectively. Another 39% of samples were wrongly identified as having the incorrect structure proposed (false negative) while only 2% were wrongly verified as the correct structure (false positive). (Figure 2)

One Day Open Access Structure Verification Results

• 116 samples
• Various experiments—from simple ¹H to comprehensive 2D sets
• Wide variety of molecules
  • MW 126–1168 Da
  • Heteroatoms: F, P
  • Different heterocycles

Upon investigation of the two samples that gave false positive results, Emine and her team learned that one was a mixture containing the proposed structure, for which only a 1D ¹H spectrum was recorded. The peaks corresponding to the proposed structure had been correctly identified and assigned, giving the positive result. The other false positive sample was verified as the incorrect positional isomer resulting from a bromination reaction. While 1D ¹H and COSY spectra were recorded, the COSY spectrum is not considered, per their ASV settings.

While the proportion of false negatives is significant, these are less concerning, because they are unlikely to mislead the chemist. However, they do require the chemist to take time to review and potentially do some reanalysis of the data. They identified the causes of the 45 false negative results as: aldehyde correlation folded in HSQC (20%), broad signals (18%), chemical shift prediction issues (11%), incorrect integration (9%), large molecule (7%), rotamers (7%), fluorine couplings not recognized (6%), and dilute sample (4%). (Figure 3)

Additionally, reviewing the false negative results is significantly faster than manually verifying structures from raw data. Dorina Kotoni, who uses this open access system for her work on the pharmaceutical development side of Novartis, testifies to this.

The structure elucidations that the CSI team undertake are often long (up to several weeks) and complex. Consequently, they want to spend as little time as possible verifying known structures. Dorina reports that the ASV embedded in the open access NMR workflow helps make this a reality.

She estimates that for manual verification an experienced analyst spends an average of 20 minutes processing and interpreting NMR data per submission. Whereas when the ASV engine successfully delivers a structure confirmation result, the analyst spends only a minute or two on review, representing a 90% time savings. When minor manual corrections are necessary, the analyst may spend 5 minutes on the analysis—less than half the time required for manual analysis. Even when ASV does not generate a result or the data processing is incomplete, it still saves the analyst time because the spectra are already referenced and processed. (Figure 4)

In the initial pilot of ASV in Novartis’ development teams, using 55 samples, 45% were completely verified by ASV, 28% had only minor problems, and 27% failed verification. After about 2.5 years of the software “learning” as more data was fed into it, the rate of complete verification was approximately 65%. (Figure 5)

Future Enhancements to ASV at Novartis

From here, Novartis’ AXS NMR team are looking to further optimize the accuracy of their ASV to provide even more time savings for their chemists. They are doing this by addressing the causes of the false results that they identified in this analysis.

Within the organization, they are working on enhancing NMR measurements, including calibration of NMR measurements (e.g., increasing the sweep width in F1 for HSQC of suspected aldehydes) and implementing standardized experiment setups. They are also continuing to enrich the prediction database with new in-house molecular data and the inclusion of diverse chemical structures to further improve prediction accuracy. Finally, they are testing v2024.1 of NMR Workbook Suite, which contains features they worked collaboratively with ACD/Labs on to address the issues with ¹⁹F couplings and integration. Based on the results of their one-day analysis, they project that these steps will significantly increase the accuracy of their ASV system. (Figure 6)

Figure 6. Projected ASV results upon addressing the causes of avoidable false negatives.

Download the case study to read offline.

Download Case Study