Getting ionizable lipid structures from online catalogs
Author
Akshay Balsubramani
Online catalogs of chemical structures
The building blocks of an ionizable lipid are fragments of different moieties in the lipid. These fragments are shared in common across combinatorial libraries, providing insight and control into their variations.
In the papers in which they appear, the fragments are typically dictated by synthesis or other resource constraints. For in silico AI/ML, we often want to mine the SMILES and lipids corresponding to known structures, which is a daunting task given the number that have been studied. Scouring the web for catalogs and literature online is the only way to keep up with such structures.
We go through the process of mining two such catalogs – Broadpharm and Medchem – which together cover the vast majority of available fragments and structures.
These cover a large fraction of the available literature, including structures that are not under patent. We will scrape each of these catalogs and combine them into a data frame that is the result of this notebook.
Procedure
In each case, getting the necessary structures from the catalogs is a two-step process:
Retrieve structure images from the catalog: This is done by scraping the catalog for the internal IDs of chemical structures. These correspond to structure images in the catalog. Each image is then downloaded from the catalog and saved to a local directory. The implementation below takes a URL and returns a data frame of the structures in the catalog.
CODE
import requests, osfrom urllib.parse import urlparsedef scrape_url(url):""" Scrapes the HTML content from a given URL and returns it as a string. Args: url (str): The URL to scrape Returns: str: The HTML content of the page Raises: ValueError: If the URL is invalid requests.RequestException: If the request fails """# Validate URL parsed_url = urlparse(url)ifnot parsed_url.scheme ornot parsed_url.netloc:raiseValueError("Invalid URL. Please provide a complete URL including http:// or https://")# Set a user agent to mimic a browser request headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36' }# Make the requesttry: response = requests.get(url, headers=headers, timeout=10) response.raise_for_status() # Raise an exception for 4XX/5XX responsesreturn response.textexcept requests.exceptions.RequestException as e:raise requests.RequestException(f"Failed to retrieve the webpage: {e}")def retrieve_chemstructure_images(IDs, names, imageID_to_url, name_to_path): img_path = {}for i inrange(len(IDs)):iflen(names[i]) ==0:continue cpd_name = names[i] image_url = imageID_to_url(IDs[i])if'/'in cpd_name: cpd_name = cpd_name.replace('/', '-') image_path = name_to_path(cpd_name)ifnot os.path.exists(image_path): img_data = requests.get(image_url).contentprint(f'Downloading {image_url} to {image_path}')withopen(image_path, 'wb') as handler: handler.write(img_data) img_path[cpd_name] = image_pathreturn img_path
Convert the structure images to SMILES strings: This is done using deep computer vision models that are trained for this specific task. We use the DECIMER transformer model for this purpose.
The SMILES string conversion requires some manual verification of the answers, so it is not yet fully automated. Though further automatic SMILES conversion presents some challenges, there is scope for addressing these as needed.1
CODE
from DECIMER import predict_SMILES as DECIMER_predict_SMILESdef predict_SMILES_from_images(img_path_dict):# To run this, DECIMER must be installed, requiring opencv-python and keras-preprocessingfrom DECIMER import predict_SMILES as DECIMER_predict_SMILES chem_smiles = {} i =0for cpd_name, image_path in img_path_dict.items(): SMILES_str = DECIMER_predict_SMILES(image_path)#SMILES_str = openchemie_predict_SMILES(image_path) chem_smiles[cpd_name] = SMILES_str i +=1print(i, cpd_name)return chem_smiles
Examples
A look at Broadpharm’s IL catalog
Broadpharm is one of the largest vendors for ionizable lipids in LNP design and has a detailed and organized catalog with solid coverage of the literature. The structures available there can be inspected from their website.
Get catalog IDs and corresponding chemical names of all structures
Each structure has a unique catalog ID which is necessary to retrieve it from within the catalog. But we typically want to store it under a different name – often the structure’s trademarked name, or another referent that is useful for looking it up in the literature.
We first retrieve the mapping between the catalog IDs and chemical names – a quick step that only involves some HTML traversals. BeautifulSoup is a standard HTML/XML parser that makes this process easier.
The code here is specific to the format of the Broadpharm catalog, and would change with time and other catalogs. The best way to write this code is therefore adaptively, with a code generation model.
CODE
from bs4 import BeautifulSoupimport pandas as pd# Parse the HTML data using BeautifulSoupsoup = BeautifulSoup(in_str, 'html.parser')# Initialize an empty list to store the extracted datacompound_data = []for tr in soup.find_all('tr'): td = tr.find_all('td') row = [i.text for i in td]iflen(row) !=6:pass# print(row)elif row[1] !='': compound = {'Product ID': row[0],'Name': row[1],'Molecular Structure': row[2],'Molecular Weight': row[3],'Purity': row[4],'Pricing': row[5] } compound_data.append(compound)# Convert the list of dictionaries to a Pandas DataFramecompound_df = pd.DataFrame(compound_data)IDs = compound_df['Product ID'].tolist()names = compound_df['Name'].tolist()
Using this mapping, it is easy to retrieve the images of all structures with their catalog IDs.
CODE
file_path_pfx ="../../files/LNP_catalog/"broadpharm_imageID_to_url =lambda x: f'https://broadpharm.com/web/images/mol_images/{x}.gif'# If the folder f'{file_path_pfx}medchem_lipids/' does not exist, create it.ifnot os.path.exists(f'{file_path_pfx}broadpharm_lipids/'): os.makedirs(f'{file_path_pfx}broadpharm_lipids/')broadpharm_name_to_path =lambda x: f'{file_path_pfx}broadpharm_lipids/{x}.png'img_path = retrieve_chemstructure_images(IDs, names, broadpharm_imageID_to_url, broadpharm_name_to_path)
Predict SMILES from image paths in dataframe
The next step is to predict the SMILES strings from the image paths in the dataframe, which is where the DECIMER model comes in.
(This could also be done using the openchemie toolkit, from which the image recognition model (Qian et al. 2023) has compared favorably to other methods from the literature (Rajan et al. 2024). )
Doing the same for MedChem’s catalog illustrates how catalog-specific these workflows get.
CODE
in_url ="https://www.medchemexpress.com/search.html?q=ionizable+lipid&type=inhibitors-and-agonists"# scrape the url manually to yield the string below, because the website appears to not be statically rendered.from selenium import webdriverfrom selenium.webdriver.chrome.options import Optionsoptions = Options()options.add_argument("--headless")options.add_argument("--disable-blink-features=AutomationControlled")options.add_argument("user-agent=Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) ""AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36")driver = webdriver.Chrome(options=options)driver.get("https://www.medchemexpress.com/search.html?q=ionizable+lipid&type=inhibitors-and-agonists")# driver.execute_script("Object.defineProperty(navigator, 'webdriver', {get: () => undefined})")in_str = driver.page_sourcedriver.quit()
Get catalog IDs and corresponding chemical names of all structures
Again, BeautifulSoup comes to the rescue, though this code needs to be adapted to the specific structure of the Medchem catalog this time. The code can in general be written by an LLM-based agent.
CODE
from bs4 import BeautifulSoupimport requests, osimport pandas as pd# Parse the HTML data using BeautifulSoupnew_soup = BeautifulSoup(in_str, 'html.parser')names = []IDs = []i =0for x in new_soup.find_all('li'):if x.dl isnotNone:iflen(x.dl.tr.text) >0: names.append(x.dl.tr.text.strip().split('\n')[0])else:iflen(x.dl.tr.a.contents) >=2: names.append(x.dl.tr.a.contents[1].contents[0])else: names.append(x.dl.tr.a.contents[0].contents[0])iflen(x.dl.dt.text) >0: IDs.append(x.dl.dt.text)else: IDs.append(x.dl.dt.a.contents[0]) i +=1
Armed with this mapping, we now again retrieve the images of all structures.
CODE
medchem_imageID_to_url =lambda x: f'https://file.medchemexpress.com/product_pic/{x}.gif'# If the folder f'{file_path_pfx}medchem_lipids/' does not exist, create it.ifnot os.path.exists(f'{file_path_pfx}medchem_lipids/'): os.makedirs(f'{file_path_pfx}medchem_lipids/')medchem_name_to_path =lambda x: f'{file_path_pfx}medchem_lipids/{x}.png'img_path = retrieve_chemstructure_images(IDs, names, medchem_imageID_to_url, medchem_name_to_path)
Predict SMILES from image paths in dataframe
Again we proceed very similarly to the Broadpharm catalog, using model predictions to convert the images to SMILES strings.
We can now merge all the catalogs that have been collected, collapsing duplicate entries appropriately and logging the source(s) of any structure. Doing a quick RDKit canonical-SMILES comparison during this process corrects for the indeterminacy in how the structure is represented (there is more than one valid SMILES for a molecule, e.g. depending on the choice of starting atom). So only unique structures are included in the final database.
CODE
import numpy as np, pandas as pdfile_path_pfx ="../../files/LNP_catalog/"smiles_df_paths = ['Broadpharm_smiles.tsv', 'Medchem_smiles.tsv']consolidated_smiles = {}for fname in smiles_df_paths: s = file_path_pfx + fname cat_name = s.split('_smiles')[0] consolidated_smiles[cat_name] =dict(np.array(pd.read_csv(s, sep='\t', header=None)))
CODE
from rdkit import Chemnew_df = {'Catalog': [],'Name': [],'SMILES': []}for catname in consolidated_smiles.keys(): thiscat = consolidated_smiles[catname] thiscat_list = [x for x inzip(*thiscat.items())] new_df['Name'].extend(list(thiscat_list[0])) new_df['SMILES'].extend(list(thiscat_list[1])) new_df['Catalog'].extend([catname] *len(thiscat))new_df = pd.DataFrame(new_df)new_df['SMILES'] = [x.replace('[R5]', 'N=[N+]=[N-]') for x in new_df['SMILES']]new_df['SMILES'] = [Chem.CanonSmiles(x) for x in new_df['SMILES']]# We find that the model occasionally mislabels explicitly drawn hydrogens as isotopic, so we fix that.new_df['SMILES'] = [x.replace('[2H]', '[H]') for x in new_df['SMILES']]new_df['SMILES'] = [x.replace('[3H]', '[H]') for x in new_df['SMILES']]# Consolidate so that new_df['SMILES'] are all unique. For any duplicates, combine them by concatenating the contents of each of their other columns.new_df_combined = new_df.groupby('SMILES').agg(lambda x: ' | '.join(set(x))).reset_index()
This combined set of SMILES can be further analyzed, or used to seed a virtual space.
By far the most comprehensive way to ensure broad virtual screening is to collect fragments, not structures, because an ultralarge virtual space defines combinatorially many structures from its fragments.
To mine such fragments, we turn to a growing body of literature using libraries that are combinatorially defined, where the study itself specifies fragments rather than the entire library of structures. Such studies almost always explicitly enumerate the library, and we can massively amplify their power by incorporating their fragments into ultralarge libraries for virtual screening.
Amine head fragments
We can retrieve a vast variety of amine fragments from non-LNP catalogs, for use in synthesis. We download an .sdf file directly from the Chemspace catalog at this link.
CODE
import zipfile, requestsdef local_zip_download(zip_path, output_dir): new_fname = zip_path.split('/')[-1]# Check if new_fname is a zip fileifnot new_fname.endswith('.zip'):return'' out_path = output_dir + new_fnamewith requests.get(zip_path, stream=True) as r: r.raise_for_status()withopen(out_path, "wb") as f:for chunk in r.iter_content(chunk_size=8192): f.write(chunk)return out_pathdef extract_sdf_from_zip(zip_path, output_dir, download=False):if download: zip_path = local_zip_download(zip_path, output_dir) fnames_written = []with zipfile.ZipFile(zip_path, 'r') as zip_ref:for file_name in zip_ref.namelist():if file_name.lower().endswith(".sdf"): zip_ref.extract(file_name, output_dir)print(f"Extracted: {file_name} to {output_dir}") fnames_written.append(file_name)return fnames_written
Chemspace stores these structures as a compressed .sdf file.
CODE
file_path_pfx ="../../files/LNP_catalog/"chemspace_file_path = file_path_pfx +'Chemspace_Amine_Fragments_Set.zip'# Replace with the desired local filename and extension
CODE
from rdkit import Chemfrom rdkit.Chem import PandasToolssdf_dir = file_path_pfxsdf_fnames = extract_sdf_from_zip(chemspace_file_path, sdf_dir)chemspace_sdf = PandasTools.LoadSDF(sdf_dir + sdf_fnames[0], removeHs=False)chemspace_sdf['SMILES'] = [Chem.MolToSmiles(x) for x in chemspace_sdf['ROMol']]chemspace_sdf['Name'] = chemspace_sdf['CHEMSPACE_ID']chemspace_sdf
Extracted: Chemspace_Amine_Fragments_Set.sdf to ../../files/LNP_catalog/
CHEMSPACE_ID
CHEMSPACE_URL
ID
ROMol
SMILES
Name
0
CSSS00007999857
https://chem-space.com/CSSS00007999857
CNC1CCCN(C2Cc3ccccc3C2)C1
CSSS00007999857
1
CSSS00012024712
https://chem-space.com/CSSS00012024712
Oc1cccc2c1CCCN2
CSSS00012024712
2
CSSS02018321032
https://chem-space.com/CSSS02018321032
O=S(=O)(Cc1ccc(F)cc1)N1CC2CCC(C1)N2
CSSS02018321032
3
CSSS00102954942
https://chem-space.com/CSSS00102954942
CN(C)c1nccc2c1CCNC2.Cl
CSSS00102954942
4
CSSS00133055275
https://chem-space.com/CSSS00133055275
Cc1ccc2oc(C(=O)NCC3CCCCN3)cc2c1.Cl
CSSS00133055275
...
...
...
...
...
...
...
18590
CSSS06359898475
https://chem-space.com/CSSS06359898475
Cl.FCCNC1CCOC1
CSSS06359898475
18591
CSSS00021525833
https://chem-space.com/CSSS00021525833
CC(C)(C)CN1CCC(C2CCNCC2)C1
CSSS00021525833
18592
CSSS00015926175
https://chem-space.com/CSSS00015926175
C1CN[C@H]2COC[C@H]2C1
CSSS00015926175
18593
CSSS00000685022
https://chem-space.com/CSSS00000685022
CNCC(=O)Nc1c(C)cc(C)cc1C.Cl
CSSS00000685022
18594
CSSS00027672546
https://chem-space.com/CSSS00027672546
Cc1ccc(C)c(C(=O)NC2CNCCC2C)c1.Cl
CSSS00027672546
18595 rows × 6 columns
Similar databases of amine fragments are available from other, smaller sources.
CODE
frags_file_path ='http://fchgroup.net/files/fragments/FCHGroup_fragment-like_amines.zip'sdf_fnames = extract_sdf_from_zip(frags_file_path, sdf_dir, download=True)fch_sdf = PandasTools.LoadSDF(sdf_dir + sdf_fnames[0], removeHs=False)fch_sdf['SMILES'] = [Chem.MolToSmiles(x) for x in fch_sdf['ROMol']]fch_sdf['Name'] = [f"FCH_{i}"for i inrange(fch_sdf.shape[0])]fch_sdf
Extracted: FCHGroup_fragment-like_amines.sdf to ../../files/LNP_catalog/
ID
URL
ROMol
SMILES
Name
0
https://chem-space.com/CSC000156186
Clc1ccc(CNc2cnccn2)cc1
FCH_0
1
https://chem-space.com/CSC000251315
CC(Nc1ncccn1)C1CC2CCC1C2
FCH_1
2
https://chem-space.com/CSC000156714
N#Cc1cccnc1NC1CCCC1
FCH_2
3
https://chem-space.com/CSC116272021
CC(C)(C)NCC(=O)NC(=O)NC1CCCCC1
FCH_3
4
https://chem-space.com/CSC116272401
COc1ccccc1C=CC1NC(=O)c2ccccc2N1
FCH_4
...
...
...
...
...
...
2150
https://chem-space.com/CSC116267824
O=C(NC[C@]12CNC[C@@]1(C(F)(F)F)C2)C1CCCC1
FCH_2150
2151
https://chem-space.com/CSC150710329
NC1CC12CC(NC(=O)c1cccc3c1CCN3)C2
FCH_2151
2152
https://chem-space.com/CSC116268621
Cc1cc(=O)[nH]c(NC2CCCc3c(C)cccc32)n1
FCH_2152
2153
https://chem-space.com/CSC150714667
C[C@H](NC1CCCC(F)(F)CC1)C(=O)N(C)C
FCH_2153
2154
https://chem-space.com/CSC150711165
Cn1cc(CN[C@@]23CCC[C@@H]2C3)ccc1=O
FCH_2154
2155 rows × 5 columns
Amines typically act as nucleophiles in organic chemistry and LNP chemistry, particularly when participating in synthesis. This means that the head fragments that scientists in LNP discovery typically care about are primary and secondary amines.
Primary amines
Primary amines are particularly useful reagents to have, so we devote special attention to their enumeration here for purposes of downstream use in other workflows. First, we gather a set of primary amines that are well studied as head groups in ionizable lipids, from a seminal line of work on short-RNA LNP delivery (Akinc et al. 2008).
Secondary amines typically offer less scope for versatile modification in syntheses, since they are already participating in a covalent bond with a carbon. MolSSI has a library of secondary amines that can be used for this purpose.
Taken together, these represent a hefty cross-section of amine fragments in commercial use. There is overlap with manufacturers’ databases, such as those of Enamine and WuXi. With the right access permissions, those too can be mined as necessary.
Having gathered this set, it’s useful to inspect these to get an idea of the staggering variety of drug-like amine fragments available (primarily because of their use in small-molecule drug design over the years).
CODE
smiles_list = []name_list = []molssi_amines_primary_df.rename(columns={'Canonical_SMILES': 'SMILES', 'Compound_Name': 'Name'}, inplace=True)molssi_amines_secondary_df.rename(columns={'Canonical_SMILES': 'SMILES', 'Compound_Name': 'Name'}, inplace=True)for new_df in [frags_whitehead_df, molssi_amines_primary_df, molssi_amines_secondary_df, chemspace_sdf, fch_sdf]: new_smiles_list = [Chem.CanonSmiles(x) for x in new_df['SMILES'].values] new_names_list = [f"{new_df['Name'].values[i]}"for i inrange(new_df.shape[0])]for i inrange(new_df.shape[0]):if new_smiles_list[i] in smiles_list: ndx = np.where(np.array(smiles_list) == new_smiles_list[i])[0][0] name_list[ndx] = name_list[ndx] +"|"+ new_names_list[i]else: smiles_list.append(new_smiles_list[i]) name_list.append(new_names_list[i])frags_amines_df = pd.DataFrame({'SMILES': smiles_list,'Name': name_list})frags_amines_df
SMILES
Name
0
C1CNC2NCCNC2N1
Whitehead_0
1
C1CNCCCNCCCNC1
Whitehead_1
2
C1CNCCCNCCNCCCNC1
Whitehead_2
3
C1CNCCN1
Whitehead_3
4
C1CNCCNC1
Whitehead_4|CSSS00000210015
...
...
...
26417
Fc1ccc(Nc2nnc(C3CC3)[nH]2)cc1
FCH_2137
26418
Cc1ccsc1CNC1CC[C@H]2CN(C)C[C@@H]12
FCH_2138
26419
NC1CC12CC(NC(=O)c1cccc3c1CCN3)C2
FCH_2151
26420
Cc1cc(=O)[nH]c(NC2CCCc3c(C)cccc32)n1
FCH_2152
26421
C[C@H](NC1CCCC(F)(F)CC1)C(=O)N(C)C
FCH_2153
26422 rows × 2 columns
CODE
amines_all =set(frags_amines_df["SMILES"])print(f"Number of amine fragments: {len(amines_all)}")display_df = pd.DataFrame(list(amines_all), columns=["SMILES"])mols2grid.display(display_df)