{
"cells": [
{
"attachments": {},
"cell_type": "markdown",
"id": "3c3a41da",
"metadata": {},
"source": [
"# Cell Correction"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "4dd6b54c",
"metadata": {},
"source": [
"This part shows how to correct cells in Stereopy. Optionally, there are two kinds of input to complete it:\n",
"\n",
" correcting from BGEF and mask;\n",
"\n",
" correcting from GEM and mask."
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "ec9191b8",
"metadata": {},
"source": [
"Provided with three algorithmic methods, you could determine which one to perform by setting the parameter `method`.\n",
"\n",
"1. `method=GMM` is based on GMM (Gaussian Mixture Model) algorithm, which performs cell correction using both gene expression matrix and spatial information, with much time and memory consumption. Multi-processing would be used if set `process_count` to more than 1.\n",
"2. `method=FAST` performs correction based on the distance between spot and centroid of the cell, when the distance is less than adjusting threshold, the spot is considered to belong to the cell. It only supports single process and single threading.\n",
"3. `method=EDM` is based on EDM (Euclidean Distance Map) algorithm, which performs correction using mask image, out of cell segmentation. We highly recommnend EDM method that is used by default. Multithreading would be on if set `process_count` to more than 1.\n",
"\n",
"More details refer to [API](../content/stereo.tools.cell_correct.cell_correct.html)."
]
},
{
"cell_type": "markdown",
"id": "d0a816c0",
"metadata": {},
"source": [
"\n",
"\n",
"**Attention**\n",
"\n",
"In order to interpret the algorithm of cell correction clearly, the parameter of which method to use has been changed. But nothing to do with the algorithms themselves.\n",
"\n",
"* `fast=False` → `method='GMM'`\n",
"* `fast='v1'` → `method='FAST'`\n",
"* `fast='v2'` → `method='EDM'`\n",
"\n",
"
"
]
},
{
"cell_type": "markdown",
"id": "357768e9",
"metadata": {},
"source": [
"\n",
"\n",
"**Note**\n",
"\n",
"The images used here must have been calibrated.\n",
"\n",
"
"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "dbe4a37d",
"metadata": {},
"source": [
"EDM algorithm is expected to eliminate the overlapping influence as much as possible, you can obvisouly learn from the following image that there is almost no overlap after cell correcting."
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "b0f5c9b3",
"metadata": {},
"source": [
""
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "3c62bd59",
"metadata": {},
"source": [
"## Correcting from BGEF and mask"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "c69495cd",
"metadata": {},
"source": [
"On this way, you should specify the path of BGEF by `bgef_path`, the path of mask by `mask_path` and the output path to save corrected result by `out_dir`. \n",
"\n",
"Cell correction dafaults to return a StereoExpData object, if set `only_save_result` to `True`, only return the path of CGEF after correcting.\n",
"\n",
"If you have no mask currently, you can generate it from ssDNA image, refer to [Cell Segmentation](https://stereopy.readthedocs.io/en/latest/Tutorials/Cell_Segmentation.html)."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "df9f2827",
"metadata": {},
"outputs": [],
"source": [
"from stereo.tools.cell_correct import cell_correct\n",
"\n",
"bgef_path = \"SS200000135TL_D1.raw.gef\"\n",
"mask_path = \"SS200000135TL_D1_mask.tif\"\n",
"out_dir = \"cell_correct_result\"\n",
"\n",
"data = cell_correct(\n",
" out_dir=out_dir,\n",
" bgef_path=bgef_path,\n",
" mask_path=mask_path,\n",
" only_save_result=False,\n",
" fast='EDM'\n",
" )"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "df52d966",
"metadata": {},
"source": [
"Output directory includes such files:\n",
"\n",
"1. `.raw.cellbin.gef` - the CGEF without correcting, generated from BGEF and mask;\n",
"2. `.adjusted.gem` - the gem after correction;\n",
"3. `.adjusted.cellbin.gef` - the CGEF after correcting, generated from the `.adjusted.gem`;\n",
"4. `err.log` - records the cells which cannot be corrected and not contained in `.adjusted.gem` and `.adjusted.cellbin.gef`."
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "c7bc80b5",
"metadata": {},
"source": [
"## Correcting from GEM and mask"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "1774d8ec",
"metadata": {},
"source": [
"In this way, you should also specify the path of BGEF by `bgef_path`, the path of mask by `mask_path` and the output path to save corrected result by `out_dir`. \n",
"\n",
"In output directory, the file named `*.bgef` is generated form mask."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "ca4f7688",
"metadata": {},
"outputs": [],
"source": [
"from stereo.tools.cell_correct import cell_correct\n",
"\n",
"gem_path = \"SS200000135TL_D1.cellbin.gem\"\n",
"mask_path = \"SS200000135TL_D1_mask.tif\"\n",
"out_dir = \"cell_correct_result\"\n",
"\n",
"data = cell_correct(\n",
" out_dir=out_dir,\n",
" gem_path=gem_path,\n",
" mask_path=mask_path,\n",
" only_save_result=False,\n",
" fast='EDM'\n",
" )"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "c6ea4543",
"metadata": {},
"source": [
"## Running on Jupyter Notebook"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "10c46cf4",
"metadata": {},
"source": [
"Notebook can not support multiprocess directly, we recommend following two steps to improve performance.\n",
"\n",
"Firstly, write the source code in a python file by command `%%writefile`."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "7254109c",
"metadata": {},
"outputs": [],
"source": [
"%%writefile temp.py\n",
"from stereo.tools.cell_correct import cell_correct\n",
"\n",
"bgef_path = \"SS200000135TL_D1.raw.gef\"\n",
"mask_path = \"SS200000135TL_D1_mask.tif\"\n",
"out_dir = \"cell_correct_result\"\n",
"\n",
"data = cell_correct(\n",
" out_dir=out_dir,\n",
" bgef_path=bgef_path,\n",
" mask_path=mask_path,\n",
" process_count=10,\n",
" only_save_result=False,\n",
" fast='GMM'\n",
" )"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "c6fdc5ca",
"metadata": {},
"source": [
"Secondly, run the file by command `%run`:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "0cf7dfab",
"metadata": {},
"outputs": [],
"source": [
"%run temp.py"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "a0839ddc",
"metadata": {},
"source": [
"\n",
"\n",
"**Note**\n",
"\n",
"We strongly suggest you output in CGEF format for subsequent analysis. When you transform CGEF to CGEM for intuitive understantding, there will be a small quantity of genes which are lost because of the algorithm, related to cell boarder. If you are concerned about the lost, just make a comparasion on the gene expression amount of the missing part.\n",
"\n",
"
"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "e7e0cf7e",
"metadata": {},
"source": [
"## Performance"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "e285f362",
"metadata": {},
"source": [
"Take a GEF which contains 56204 cells and 24661 genes as an example."
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "d91860b9",
"metadata": {},
"source": [
"Machine configuration as below:"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "b3ce7665",
"metadata": {},
"source": [
"|physical cores |logic cores |memory |\n",
"| --- | --- | --- |\n",
"|12 |48 |250G |"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "6efc1d70",
"metadata": {},
"source": [
"Comparision of performance:"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "b8c85741",
"metadata": {},
"source": [
"`fast='GMM'`"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "d6baf5a3",
"metadata": {},
"source": [
"|process |memory(max) |cpu |time |\n",
"| --- | --- | --- | --- |\n",
"|10 |120G |2330% |2h38m |"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "cc40c942",
"metadata": {},
"source": [
"`fast='FAST'` (only support single process)"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "b023c1d5",
"metadata": {},
"source": [
"|process |memory(max) |cpu |time |\n",
"| --- | --- | --- | --- |\n",
"|1 |41G |100% |26m |"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "d0e2c111",
"metadata": {},
"source": [
"`fast='EDM'` (single process)"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "06876fec",
"metadata": {},
"source": [
"|process |memory(max) |cpu |time |\n",
"| --- | --- | --- | --- |\n",
"|1 |15G |100% |9m |"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.6.2"
},
"vscode": {
"interpreter": {
"hash": "70dbeb2a90198859cd91b6ea0f3adc73d66939fe301617b631d99dfc954c0323"
}
}
},
"nbformat": 4,
"nbformat_minor": 5
}