文学式编程-nbdev入门教程

nbdev是一个使用Jupyter Notebook进行多模块软件开发的辅助工具,能够将多个Notebook组装为一个大型的软件系统,自动生成代码和文档,并可以在Notebook中进行交互运行和探索性测试。html

nbdev主要完成下面的功能:git

  • 创建python开发框架和git repo版本。
  • 在notebook中的cell中使用#export标记为输出代码区块。
  • 使用库函数notebook2script()将#export标记的代码转化为python库*.py。
  • 所生成的代码放在project name的目录下,能够在notebook中引用或者直接运行。github

  • 例程项目,用于nCoV分析(初级阶段):https://github.com/openthings/anti2020ncov

一、安装

nbdev已经加入 PyPI,使用下面的命令安装:框架

pip install nbdev

对于 editable install,使用:less

git clone https://github.com/fastai/nbdev
pip install -e nbdev

二、快速开始

2.1 建立项目

开始本身的 project, 点击: nbdev template (会要求登陆到 GitHub )。填写要求的信息,而后点击 Create repository from template, 新的 GitHub repo将会在你的帐户下建立出来。ide

注意: 项目的名称将成为 Python package的名称,由 nbdev 自动建立。建议选择短的所有小写并且没有链接线的名称(下划线能够)。函数

如今,打开终端,将该 repo 拉去到本地(git clone ...),可使用Jupyter里面建立的终端窗口。工具

可选方式,可使用nbdev的客户端命令 nbdev_new 来建立nbdev project,而且初始化git repository到本地,不会建立 github repo。测试

2.2 设置参数

下一步,编辑 settings.ini 文件。

该文件包含全部的打包你的库的信息所以不须要改变 setup.py 文件,这是由模版工程建立的。基本结构 (that can be personalized provided you change the relevant information in settings.ini) 是 repo的根目录将包含你的 notebooks, 目录 docs 包含文档是自动建立的,可用于 jekyll-powered 站点。由于 GitHub Pages supports Jekyll, 能够将该文档放到 GitHub ,不须要作任何额外的设置。

你的 settings.ini 放在 nbdev,用于任何须要的配置信息。若是编辑了该文件,运行命令 nbdev_build_lib (安装 nbdev时自动安装)。 你将发现有了一个新的新的目录,按照 lib_name 在 settings.ini中的设置。

如今运行jupyter notebook, 点击 00_core.ipynb。 这是你建立的第一个模块。能够在notebook中建立多个Jupyter cells。 在须要包含在python模块的cell最前面加上 #export ,必须是第一行。

注意,Note that for any new .ipynb that you create, you will also need to type #default_exp target_module_name, which will define the name of the generated module (lib_name/target_module_name.py).

In the last cell of your notebook, you can then run:

from nbdev.export import *
notebook2script()
Converted 00_export.ipynb.
Converted 01_sync.ipynb.
Converted 02_showdoc.ipynb.
Converted 03_export2html.ipynb.
Converted 04_test.ipynb.
Converted 05_merge.ipynb.
Converted 06_cli.ipynb.
Converted 07_clean.ipynb.
Converted 99_search.ipynb.
Converted index.ipynb.
Converted tutorial.ipynb.

Or in the command line, you can run:

nbdev_build_lib

as long as you are somewhere in the folder where you are developing your library. Either of these will do the same thing: update your module to include all exported cells in your notebook.

To enable documentation in your GitHub repo, click 'Settings' on the main repo page, scroll down to 'GitHub Pages', and under 'Source' choose 'master branch /docs folder'. GitHub will then show you a link to your working documentation site.

Finally, edit index.ipynb. This will be converted into your projects README file, and will also be the index for your documentation (the page you're reading right now actually comes from an index.ipynb file!) You can use the module you just exported in this library, which means you can show real working code, and actual outputs. Once you have everything as you want it, run nbdev_build_docs in the terminal. This will export HTML versions of your notebooks to the docs directory, and will create hyperlinks for any words in backticks (as long as they exist in your module). It will also create a menu for all notebooks you have created, and a table of contents for each.

Note if using a subdirectory to contain .ipynb files instead of the project root

If you have set the parameter nbs_path to be anything other than the project root, you will not be able to import your generated modules without an extra step:

  • either install these modules locally, as their relative import will take you beyond the top-level package, which can be done by running pip install -e . in the project root, to install the modules to your environment in editable mode.
  • or make a simlink in your notebook folder to the library folder, which can be done by running ln -s lib_path lib_name (adjust lib_path and lib_name to your use case).

Additional functionality

There's a lot of functionality in nbdev; see the docs for each module in the sidebar to learn about all the features. Here we'll briefly highlight a couple.

Adding your project to pypi

If you want people to be able to install your project by just typing pip install your-project then you need to upload it to pypi. The good news is, we've already created a fully pypi compliant installer for your project! So all you need to do is register at pypi, if you haven't previously done so, and then create a file called ~/.pypirc with your login details. It should have these contents:

[pypi]
username = your_pypi_username
password = your_pypi_password

Another thing you will need is twine, so you should run once

pip install twine

To upload your project to pypi, just type make pypi in your project root directory. Once it's complete, a link to your project on pypi will be printed.

NB: make sure you increment the version number in settings.py each time you want to push a new release to pypi.

Avoiding and handling git conflicts

Jupyter Notebooks can cause challenges with git conflicts, but life becomes much easier when you use nbdev. As a first step, run nbdev_install_git_hooks in the terminal from your project folder. This will set up git hooks which will remove metadata from your notebooks when you commit, greatly reducing the chance you have a conflict.

But if you do get a conflict, simply run nbdev_fix_merge filename.ipynb. This will replace any conflicts in cell outputs with your version, and if there are conflicts in input cells, then both cells will be included in the merged file, along with standard conflict markers (e.g. =====). Then you can open the notebook in Jupyter and choose which version to keep.

Using nbdev as part of your CI

You can use GitHub actions to leverage the functionality of nbdev and easily make a CI that:

  • check the notebooks are readable (with nbdev_read_nbs)
  • check the notebooks have been cleaned of needless metadata to avoid merge conflicts (with nbdev_clean_nbs)
  • check there is no diff between the notebooks and the exported library (with nbdev_diff_nbs)
  • run the tests in your notebooks (with nbdev_test_nbs)

The template contains a basic CI that uses the four points above, edit the file .github/workflows/main.yml to your liking and comment out the parts you don't want.

Math equation support

nbdev supports equations (we use the excellent KaTeX library). Enable it with use_math: true in your _config.yml (it's enabled by default). You can include math in your notebook's documentation using the following methods.

Using $$, e.g.:

$$\sum_{i=1}^{k+1}i$$

Which is rendered as:

$$\sum_{i=1}^{k+1}i$$

Using $, e.g.:

This version is diplayed inline: $\sum_{i=1}^{k+1}i$ . You can include text before and after.

Which is rendered as:

This version is diplayed inline: $\sum_{i=1}^{k+1}i$ . You can include text before and after.

 

更多参考: