class: misk-title-slide <a href="https://github.com/misk-data-science/misk-packages"><img style="position: absolute; top: 0; right: 0; border: 0;" src="https://s3.amazonaws.com/github/ribbons/forkme_right_darkblue_121621.png" alt="Fork me on GitHub"></a> <br><br><br><br> # .font120[Delivering Data Science Products<br>via Packages] --- class: misk-section-slide <br><br><br><br><br><br><br> .bold.font250[Introduction] --- # What are packages? * Packages are the fundamental units of reproducible R and Python code. * They include reusable components, the documentation that describes how to use them, requirements to ensure the user can apply them and tests to ensure consistent and reliable functionality. <div class="note"> <p>.font80[ People often use the terms “package” and “library” synonymously.</p> <ul> <li><p><strong>package</strong>: generally refers to source code that is bundled up in a way that a package manager can host. <a href="https://pypi.org/">PyPI</a> and <a href="https://cran.r-project.org/">CRAN</a> are the two primary public package managers for Python and R. When you <code>pip install pkg</code> (Python) or <code>install.packages("pkg")</code> (R) you are installing the <strong>pkg</strong> package from a package manager onto a computer.</p></li> <li><p><strong>library</strong>: generally refers to a centralized location on an operating system where installed package source code resides and can be imported into a current session (i.e. /usr/lib/R/library). When you use <code>pip list</code> (Python) or <code>installed.packages()</code> (R) you will see a list of <em>installed</em> packages, which we refer to as libraries. When you run <code>sys.path</code> (Python) or <code>.libPaths()</code> (R) you will get the path to the library where your installed packages are stored. ]</p></li> </ul> </div> --- # Packaging .pull-left.font120[ .bold[Why?] - To share with others - Reproducibility - Track changes - Communication - Reliability - Production ] -- .pull-left.font120[ .bold[Why not?] - Exploratory analysis - One-off projects - Small scripts <br> .center.font90.red[___Although the above type of work may not justify a package, it often still justifies good software engineering practices such as modularity and unit tests!___] ] --- # Things to be aware of * Several approaches and strategies to writing packages * This course demonstrates commonly accepted best practices * __Objective__: provide you with a short runway to writing packages as quickly as possible while following commonly used best practices -- <br> <div class="note"> <p>This is a lot to learn, but don’t feel overwhelmed. Start with a minimal subset of useful features and build up over time.</p> </div> --- # Conventions used throughout .font120[ - Python: <svg style="height:0.8em;top:.04em;position:relative;" viewBox="0 0 448 512"><path d="M439.8 200.5c-7.7-30.9-22.3-54.2-53.4-54.2h-40.1v47.4c0 36.8-31.2 67.8-66.8 67.8H172.7c-29.2 0-53.4 25-53.4 54.3v101.8c0 29 25.2 46 53.4 54.3 33.8 9.9 66.3 11.7 106.8 0 26.9-7.8 53.4-23.5 53.4-54.3v-40.7H226.2v-13.6h160.2c31.1 0 42.6-21.7 53.4-54.2 11.2-33.5 10.7-65.7 0-108.6zM286.2 404c11.1 0 20.1 9.1 20.1 20.3 0 11.3-9 20.4-20.1 20.4-11 0-20.1-9.2-20.1-20.4.1-11.3 9.1-20.3 20.1-20.3zM167.8 248.1h106.8c29.7 0 53.4-24.5 53.4-54.3V91.9c0-29-24.4-50.7-53.4-55.6-35.8-5.9-74.7-5.6-106.8.1-45.2 8-53.4 24.7-53.4 55.6v40.7h106.9v13.6h-147c-31.1 0-58.3 18.7-66.8 54.2-9.8 40.7-10.2 66.1 0 108.6 7.6 31.6 25.7 54.2 56.8 54.2H101v-48.8c0-35.3 30.5-66.4 66.8-66.4zm-6.7-142.6c-11.1 0-20.1-9.1-20.1-20.3.1-11.3 9-20.4 20.1-20.4 11 0 20.1 9.2 20.1 20.4s-9 20.3-20.1 20.3z"/></svg> - R: <svg style="height:0.8em;top:.04em;position:relative;" viewBox="0 0 581 512"><path d="M581 226.6C581 119.1 450.9 32 290.5 32S0 119.1 0 226.6C0 322.4 103.3 402 239.4 418.1V480h99.1v-61.5c24.3-2.7 47.6-7.4 69.4-13.9L448 480h112l-67.4-113.7c54.5-35.4 88.4-84.9 88.4-139.7zm-466.8 14.5c0-73.5 98.9-133 220.8-133s211.9 40.7 211.9 133c0 50.1-26.5 85-70.3 106.4-2.4-1.6-4.7-2.9-6.4-3.7-10.2-5.2-27.8-10.5-27.8-10.5s86.6-6.4 86.6-92.7-90.6-87.9-90.6-87.9h-199V361c-74.1-21.5-125.2-67.1-125.2-119.9zm225.1 38.3v-55.6c57.8 0 87.8-6.8 87.8 27.3 0 36.5-38.2 28.3-87.8 28.3zm-.9 72.5H365c10.8 0 18.9 11.7 24 19.2-16.1 1.9-33 2.8-50.6 2.9v-22.1z"/></svg> ] <div class="tip"> <p>Signifies a tip or suggestion</p> </div> <div class="note"> <p>Signifies a general note</p> </div> <div class="warning"> <p>Signifies a warning or caution</p> </div> --- # Exercises <br> 1. Skim the table of contents for the following "official guides" to <svg style="height:0.8em;top:.04em;position:relative;" viewBox="0 0 581 512"><path d="M581 226.6C581 119.1 450.9 32 290.5 32S0 119.1 0 226.6C0 322.4 103.3 402 239.4 418.1V480h99.1v-61.5c24.3-2.7 47.6-7.4 69.4-13.9L448 480h112l-67.4-113.7c54.5-35.4 88.4-84.9 88.4-139.7zm-466.8 14.5c0-73.5 98.9-133 220.8-133s211.9 40.7 211.9 133c0 50.1-26.5 85-70.3 106.4-2.4-1.6-4.7-2.9-6.4-3.7-10.2-5.2-27.8-10.5-27.8-10.5s86.6-6.4 86.6-92.7-90.6-87.9-90.6-87.9h-199V361c-74.1-21.5-125.2-67.1-125.2-119.9zm225.1 38.3v-55.6c57.8 0 87.8-6.8 87.8 27.3 0 36.5-38.2 28.3-87.8 28.3zm-.9 72.5H365c10.8 0 18.9 11.7 24 19.2-16.1 1.9-33 2.8-50.6 2.9v-22.1z"/></svg> and <svg style="height:0.8em;top:.04em;position:relative;" viewBox="0 0 448 512"><path d="M439.8 200.5c-7.7-30.9-22.3-54.2-53.4-54.2h-40.1v47.4c0 36.8-31.2 67.8-66.8 67.8H172.7c-29.2 0-53.4 25-53.4 54.3v101.8c0 29 25.2 46 53.4 54.3 33.8 9.9 66.3 11.7 106.8 0 26.9-7.8 53.4-23.5 53.4-54.3v-40.7H226.2v-13.6h160.2c31.1 0 42.6-21.7 53.4-54.2 11.2-33.5 10.7-65.7 0-108.6zM286.2 404c11.1 0 20.1 9.1 20.1 20.3 0 11.3-9 20.4-20.1 20.4-11 0-20.1-9.2-20.1-20.4.1-11.3 9.1-20.3 20.1-20.3zM167.8 248.1h106.8c29.7 0 53.4-24.5 53.4-54.3V91.9c0-29-24.4-50.7-53.4-55.6-35.8-5.9-74.7-5.6-106.8.1-45.2 8-53.4 24.7-53.4 55.6v40.7h106.9v13.6h-147c-31.1 0-58.3 18.7-66.8 54.2-9.8 40.7-10.2 66.1 0 108.6 7.6 31.6 25.7 54.2 56.8 54.2H101v-48.8c0-35.3 30.5-66.4 66.8-66.4zm-6.7-142.6c-11.1 0-20.1-9.1-20.1-20.3.1-11.3 9-20.4 20.1-20.4 11 0 20.1 9.2 20.1 20.4s-9 20.3-20.1 20.3z"/></svg> packaging. Do not worry about reading the details but realize that these are good sources to start looking at when you have questions about certain aspects of packaging. - <svg style="height:0.8em;top:.04em;position:relative;" viewBox="0 0 581 512"><path d="M581 226.6C581 119.1 450.9 32 290.5 32S0 119.1 0 226.6C0 322.4 103.3 402 239.4 418.1V480h99.1v-61.5c24.3-2.7 47.6-7.4 69.4-13.9L448 480h112l-67.4-113.7c54.5-35.4 88.4-84.9 88.4-139.7zm-466.8 14.5c0-73.5 98.9-133 220.8-133s211.9 40.7 211.9 133c0 50.1-26.5 85-70.3 106.4-2.4-1.6-4.7-2.9-6.4-3.7-10.2-5.2-27.8-10.5-27.8-10.5s86.6-6.4 86.6-92.7-90.6-87.9-90.6-87.9h-199V361c-74.1-21.5-125.2-67.1-125.2-119.9zm225.1 38.3v-55.6c57.8 0 87.8-6.8 87.8 27.3 0 36.5-38.2 28.3-87.8 28.3zm-.9 72.5H365c10.8 0 18.9 11.7 24 19.2-16.1 1.9-33 2.8-50.6 2.9v-22.1z"/></svg>: [Creating R Extensions](https://cran.r-project.org/doc/manuals/R-exts.html#Creating-R-packages) - <svg style="height:0.8em;top:.04em;position:relative;" viewBox="0 0 448 512"><path d="M439.8 200.5c-7.7-30.9-22.3-54.2-53.4-54.2h-40.1v47.4c0 36.8-31.2 67.8-66.8 67.8H172.7c-29.2 0-53.4 25-53.4 54.3v101.8c0 29 25.2 46 53.4 54.3 33.8 9.9 66.3 11.7 106.8 0 26.9-7.8 53.4-23.5 53.4-54.3v-40.7H226.2v-13.6h160.2c31.1 0 42.6-21.7 53.4-54.2 11.2-33.5 10.7-65.7 0-108.6zM286.2 404c11.1 0 20.1 9.1 20.1 20.3 0 11.3-9 20.4-20.1 20.4-11 0-20.1-9.2-20.1-20.4.1-11.3 9.1-20.3 20.1-20.3zM167.8 248.1h106.8c29.7 0 53.4-24.5 53.4-54.3V91.9c0-29-24.4-50.7-53.4-55.6-35.8-5.9-74.7-5.6-106.8.1-45.2 8-53.4 24.7-53.4 55.6v40.7h106.9v13.6h-147c-31.1 0-58.3 18.7-66.8 54.2-9.8 40.7-10.2 66.1 0 108.6 7.6 31.6 25.7 54.2 56.8 54.2H101v-48.8c0-35.3 30.5-66.4 66.8-66.4zm-6.7-142.6c-11.1 0-20.1-9.1-20.1-20.3.1-11.3 9-20.4 20.1-20.4 11 0 20.1 9.2 20.1 20.4s-9 20.3-20.1 20.3z"/></svg>: [Python Packaging User Guide](https://packaging.python.org/) 2. Identify two popular <svg style="height:0.8em;top:.04em;position:relative;" viewBox="0 0 581 512"><path d="M581 226.6C581 119.1 450.9 32 290.5 32S0 119.1 0 226.6C0 322.4 103.3 402 239.4 418.1V480h99.1v-61.5c24.3-2.7 47.6-7.4 69.4-13.9L448 480h112l-67.4-113.7c54.5-35.4 88.4-84.9 88.4-139.7zm-466.8 14.5c0-73.5 98.9-133 220.8-133s211.9 40.7 211.9 133c0 50.1-26.5 85-70.3 106.4-2.4-1.6-4.7-2.9-6.4-3.7-10.2-5.2-27.8-10.5-27.8-10.5s86.6-6.4 86.6-92.7-90.6-87.9-90.6-87.9h-199V361c-74.1-21.5-125.2-67.1-125.2-119.9zm225.1 38.3v-55.6c57.8 0 87.8-6.8 87.8 27.3 0 36.5-38.2 28.3-87.8 28.3zm-.9 72.5H365c10.8 0 18.9 11.7 24 19.2-16.1 1.9-33 2.8-50.6 2.9v-22.1z"/></svg> packages and explain why packaging these capabilities is beneficial. 3. Identify two popular <svg style="height:0.8em;top:.04em;position:relative;" viewBox="0 0 448 512"><path d="M439.8 200.5c-7.7-30.9-22.3-54.2-53.4-54.2h-40.1v47.4c0 36.8-31.2 67.8-66.8 67.8H172.7c-29.2 0-53.4 25-53.4 54.3v101.8c0 29 25.2 46 53.4 54.3 33.8 9.9 66.3 11.7 106.8 0 26.9-7.8 53.4-23.5 53.4-54.3v-40.7H226.2v-13.6h160.2c31.1 0 42.6-21.7 53.4-54.2 11.2-33.5 10.7-65.7 0-108.6zM286.2 404c11.1 0 20.1 9.1 20.1 20.3 0 11.3-9 20.4-20.1 20.4-11 0-20.1-9.2-20.1-20.4.1-11.3 9.1-20.3 20.1-20.3zM167.8 248.1h106.8c29.7 0 53.4-24.5 53.4-54.3V91.9c0-29-24.4-50.7-53.4-55.6-35.8-5.9-74.7-5.6-106.8.1-45.2 8-53.4 24.7-53.4 55.6v40.7h106.9v13.6h-147c-31.1 0-58.3 18.7-66.8 54.2-9.8 40.7-10.2 66.1 0 108.6 7.6 31.6 25.7 54.2 56.8 54.2H101v-48.8c0-35.3 30.5-66.4 66.8-66.4zm-6.7-142.6c-11.1 0-20.1-9.1-20.1-20.3.1-11.3 9-20.4 20.1-20.4 11 0 20.1 9.2 20.1 20.4s-9 20.3-20.1 20.3z"/></svg> packages and explain why packaging these capabilities is beneficial. --- class: misk-section-slide <br><br><br><br><br><br><br> .bold.font250[Package Structure] --- # Basic required files .pull-left[ Minimum files required or highly suggested: * Package metadata * Source code directory * Test directory * Documentation directory * License file * Changelog * README * Other various config or requirement files ] .pull-right[ ```sh . ├── package metadata file ├── source code directory │ ├── script 1 │ ├── script 2 │ └── script n ├── test directory │ ├── script 1 │ ├── script 2 │ └── script n ├── documentation directory │ └── various doc files ├── license file ├── changelog file ├── README └── other various config files and components ``` ] --- class: inverse, hide-logo <br><br><br><br><br> .center.white.font200[Let's assume a package called .bold.red[mypkg]] --- # <svg style="height:0.8em;top:.04em;position:relative;fill:white;" viewBox="0 0 581 512"><path d="M581 226.6C581 119.1 450.9 32 290.5 32S0 119.1 0 226.6C0 322.4 103.3 402 239.4 418.1V480h99.1v-61.5c24.3-2.7 47.6-7.4 69.4-13.9L448 480h112l-67.4-113.7c54.5-35.4 88.4-84.9 88.4-139.7zm-466.8 14.5c0-73.5 98.9-133 220.8-133s211.9 40.7 211.9 133c0 50.1-26.5 85-70.3 106.4-2.4-1.6-4.7-2.9-6.4-3.7-10.2-5.2-27.8-10.5-27.8-10.5s86.6-6.4 86.6-92.7-90.6-87.9-90.6-87.9h-199V361c-74.1-21.5-125.2-67.1-125.2-119.9zm225.1 38.3v-55.6c57.8 0 87.8-6.8 87.8 27.3 0 36.5-38.2 28.3-87.8 28.3zm-.9 72.5H365c10.8 0 18.9 11.7 24 19.2-16.1 1.9-33 2.8-50.6 2.9v-22.1z"/></svg> file structure .pull-left[ Unique items here are: - __DESCRIPTION__: This is the file that contains the primary package metadata. - __R/__: The R directory is where all the source code resides. - __NEWS__: It is common convention in many R packages to title the changelog as "NEWS". ] .pull-right[ ```sh mypkg ├── DESCRIPTION ├── R/ ├── tests/ ├── docs/ ├── LICENSE ├── NEWS └── README ``` ] --- # <svg style="height:0.8em;top:.04em;position:relative;fill:white;" viewBox="0 0 448 512"><path d="M439.8 200.5c-7.7-30.9-22.3-54.2-53.4-54.2h-40.1v47.4c0 36.8-31.2 67.8-66.8 67.8H172.7c-29.2 0-53.4 25-53.4 54.3v101.8c0 29 25.2 46 53.4 54.3 33.8 9.9 66.3 11.7 106.8 0 26.9-7.8 53.4-23.5 53.4-54.3v-40.7H226.2v-13.6h160.2c31.1 0 42.6-21.7 53.4-54.2 11.2-33.5 10.7-65.7 0-108.6zM286.2 404c11.1 0 20.1 9.1 20.1 20.3 0 11.3-9 20.4-20.1 20.4-11 0-20.1-9.2-20.1-20.4.1-11.3 9.1-20.3 20.1-20.3zM167.8 248.1h106.8c29.7 0 53.4-24.5 53.4-54.3V91.9c0-29-24.4-50.7-53.4-55.6-35.8-5.9-74.7-5.6-106.8.1-45.2 8-53.4 24.7-53.4 55.6v40.7h106.9v13.6h-147c-31.1 0-58.3 18.7-66.8 54.2-9.8 40.7-10.2 66.1 0 108.6 7.6 31.6 25.7 54.2 56.8 54.2H101v-48.8c0-35.3 30.5-66.4 66.8-66.4zm-6.7-142.6c-11.1 0-20.1-9.1-20.1-20.3.1-11.3 9-20.4 20.1-20.4 11 0 20.1 9.2 20.1 20.4s-9 20.3-20.1 20.3z"/></svg> file structure .pull-left[ Unique items here are: - __setup.py__: This is the file that contains the primary package metadata. - __src/__: The src directory is where all the source code resides. ] .pull-right[ ```sh mypkg ├── setup.py ├── src/ ├── tests/ ├── docs/ ├── LICENSE ├── CHANGELOG └── README ``` ] --- # Exercises 1. Look at the package structure for the <svg style="height:0.8em;top:.04em;position:relative;" viewBox="0 0 581 512"><path d="M581 226.6C581 119.1 450.9 32 290.5 32S0 119.1 0 226.6C0 322.4 103.3 402 239.4 418.1V480h99.1v-61.5c24.3-2.7 47.6-7.4 69.4-13.9L448 480h112l-67.4-113.7c54.5-35.4 88.4-84.9 88.4-139.7zm-466.8 14.5c0-73.5 98.9-133 220.8-133s211.9 40.7 211.9 133c0 50.1-26.5 85-70.3 106.4-2.4-1.6-4.7-2.9-6.4-3.7-10.2-5.2-27.8-10.5-27.8-10.5s86.6-6.4 86.6-92.7-90.6-87.9-90.6-87.9h-199V361c-74.1-21.5-125.2-67.1-125.2-119.9zm225.1 38.3v-55.6c57.8 0 87.8-6.8 87.8 27.3 0 36.5-38.2 28.3-87.8 28.3zm-.9 72.5H365c10.8 0 18.9 11.7 24 19.2-16.1 1.9-33 2.8-50.6 2.9v-22.1z"/></svg> package [tidyr](https://github.com/tidyverse/tidyr/) - Do all the files exist that were discussed in this module? - Take a quick glance at each of the files discussed in this section. - What additional files are present? 2. Look at the package structure for the <svg style="height:0.8em;top:.04em;position:relative;" viewBox="0 0 448 512"><path d="M439.8 200.5c-7.7-30.9-22.3-54.2-53.4-54.2h-40.1v47.4c0 36.8-31.2 67.8-66.8 67.8H172.7c-29.2 0-53.4 25-53.4 54.3v101.8c0 29 25.2 46 53.4 54.3 33.8 9.9 66.3 11.7 106.8 0 26.9-7.8 53.4-23.5 53.4-54.3v-40.7H226.2v-13.6h160.2c31.1 0 42.6-21.7 53.4-54.2 11.2-33.5 10.7-65.7 0-108.6zM286.2 404c11.1 0 20.1 9.1 20.1 20.3 0 11.3-9 20.4-20.1 20.4-11 0-20.1-9.2-20.1-20.4.1-11.3 9.1-20.3 20.1-20.3zM167.8 248.1h106.8c29.7 0 53.4-24.5 53.4-54.3V91.9c0-29-24.4-50.7-53.4-55.6-35.8-5.9-74.7-5.6-106.8.1-45.2 8-53.4 24.7-53.4 55.6v40.7h106.9v13.6h-147c-31.1 0-58.3 18.7-66.8 54.2-9.8 40.7-10.2 66.1 0 108.6 7.6 31.6 25.7 54.2 56.8 54.2H101v-48.8c0-35.3 30.5-66.4 66.8-66.4zm-6.7-142.6c-11.1 0-20.1-9.1-20.1-20.3.1-11.3 9-20.4 20.1-20.4 11 0 20.1 9.2 20.1 20.4s-9 20.3-20.1 20.3z"/></svg> package [pytest](https://github.com/pytest-dev/pytest) - Do all the files exist that were discussed in this module? - Take a quick glance at each of the files discussed in this section. - What additional files are present? --- class: misk-section-slide <br><br><br><br><br><br><br> .bold.font250[Development Workflow] --- # Typical workflow .pull-left[ .center.bold[Initial Development] 1. Identify package name 2. Create package structure 3. Enable version control 4. Setup virtual environment 5. Add new functionality (TDD style) 6. Document 7. Version control ] -- .pull-right[ .center.bold[Ongoing Development] 1. .opacity10[Identify package name] 2. .opacity10[Create package structure] 3. .opacity10[Enable version control] 4. .opacity10[Setup virtual environment] 5. Add new functionality (TDD style) 6. Document 7. Version control ] --- # Package name - Easy to remember - Follow the respective languages idiomatic approach to naming - Should not already exist. - Certain requirements we need to adhere to - <svg style="height:0.8em;top:.04em;position:relative;" viewBox="0 0 581 512"><path d="M581 226.6C581 119.1 450.9 32 290.5 32S0 119.1 0 226.6C0 322.4 103.3 402 239.4 418.1V480h99.1v-61.5c24.3-2.7 47.6-7.4 69.4-13.9L448 480h112l-67.4-113.7c54.5-35.4 88.4-84.9 88.4-139.7zm-466.8 14.5c0-73.5 98.9-133 220.8-133s211.9 40.7 211.9 133c0 50.1-26.5 85-70.3 106.4-2.4-1.6-4.7-2.9-6.4-3.7-10.2-5.2-27.8-10.5-27.8-10.5s86.6-6.4 86.6-92.7-90.6-87.9-90.6-87.9h-199V361c-74.1-21.5-125.2-67.1-125.2-119.9zm225.1 38.3v-55.6c57.8 0 87.8-6.8 87.8 27.3 0 36.5-38.2 28.3-87.8 28.3zm-.9 72.5H365c10.8 0 18.9 11.7 24 19.2-16.1 1.9-33 2.8-50.6 2.9v-22.1z"/></svg> your name can contain a `.` but not a `-` or `_` - <svg style="height:0.8em;top:.04em;position:relative;" viewBox="0 0 448 512"><path d="M439.8 200.5c-7.7-30.9-22.3-54.2-53.4-54.2h-40.1v47.4c0 36.8-31.2 67.8-66.8 67.8H172.7c-29.2 0-53.4 25-53.4 54.3v101.8c0 29 25.2 46 53.4 54.3 33.8 9.9 66.3 11.7 106.8 0 26.9-7.8 53.4-23.5 53.4-54.3v-40.7H226.2v-13.6h160.2c31.1 0 42.6-21.7 53.4-54.2 11.2-33.5 10.7-65.7 0-108.6zM286.2 404c11.1 0 20.1 9.1 20.1 20.3 0 11.3-9 20.4-20.1 20.4-11 0-20.1-9.2-20.1-20.4.1-11.3 9.1-20.3 20.1-20.3zM167.8 248.1h106.8c29.7 0 53.4-24.5 53.4-54.3V91.9c0-29-24.4-50.7-53.4-55.6-35.8-5.9-74.7-5.6-106.8.1-45.2 8-53.4 24.7-53.4 55.6v40.7h106.9v13.6h-147c-31.1 0-58.3 18.7-66.8 54.2-9.8 40.7-10.2 66.1 0 108.6 7.6 31.6 25.7 54.2 56.8 54.2H101v-48.8c0-35.3 30.5-66.4 66.8-66.4zm-6.7-142.6c-11.1 0-20.1-9.1-20.1-20.3.1-11.3 9-20.4 20.1-20.4 11 0 20.1 9.2 20.1 20.4s-9 20.3-20.1 20.3z"/></svg> your name can contain all three but not recommended - In both languages you can combine upper and lowercase letters in the name. - Examples of good names: - numpy - dplyr - pandas - keras --- # Package structure Where will the package live on your operating system? - Should be distinct from where installed packages live - Typically designate a directory inside their home directory for <svg style="height:0.8em;top:.04em;position:relative;" viewBox="0 0 581 512"><path d="M581 226.6C581 119.1 450.9 32 290.5 32S0 119.1 0 226.6C0 322.4 103.3 402 239.4 418.1V480h99.1v-61.5c24.3-2.7 47.6-7.4 69.4-13.9L448 480h112l-67.4-113.7c54.5-35.4 88.4-84.9 88.4-139.7zm-466.8 14.5c0-73.5 98.9-133 220.8-133s211.9 40.7 211.9 133c0 50.1-26.5 85-70.3 106.4-2.4-1.6-4.7-2.9-6.4-3.7-10.2-5.2-27.8-10.5-27.8-10.5s86.6-6.4 86.6-92.7-90.6-87.9-90.6-87.9h-199V361c-74.1-21.5-125.2-67.1-125.2-119.9zm225.1 38.3v-55.6c57.8 0 87.8-6.8 87.8 27.3 0 36.5-38.2 28.3-87.8 28.3zm-.9 72.5H365c10.8 0 18.9 11.7 24 19.2-16.1 1.9-33 2.8-50.6 2.9v-22.1z"/></svg> and <svg style="height:0.8em;top:.04em;position:relative;" viewBox="0 0 448 512"><path d="M439.8 200.5c-7.7-30.9-22.3-54.2-53.4-54.2h-40.1v47.4c0 36.8-31.2 67.8-66.8 67.8H172.7c-29.2 0-53.4 25-53.4 54.3v101.8c0 29 25.2 46 53.4 54.3 33.8 9.9 66.3 11.7 106.8 0 26.9-7.8 53.4-23.5 53.4-54.3v-40.7H226.2v-13.6h160.2c31.1 0 42.6-21.7 53.4-54.2 11.2-33.5 10.7-65.7 0-108.6zM286.2 404c11.1 0 20.1 9.1 20.1 20.3 0 11.3-9 20.4-20.1 20.4-11 0-20.1-9.2-20.1-20.4.1-11.3 9.1-20.3 20.1-20.3zM167.8 248.1h106.8c29.7 0 53.4-24.5 53.4-54.3V91.9c0-29-24.4-50.7-53.4-55.6-35.8-5.9-74.7-5.6-106.8.1-45.2 8-53.4 24.7-53.4 55.6v40.7h106.9v13.6h-147c-31.1 0-58.3 18.7-66.8 54.2-9.8 40.7-10.2 66.1 0 108.6 7.6 31.6 25.7 54.2 56.8 54.2H101v-48.8c0-35.3 30.5-66.4 66.8-66.4zm-6.7-142.6c-11.1 0-20.1-9.1-20.1-20.3.1-11.3 9-20.4 20.1-20.4 11 0 20.1 9.2 20.1 20.4s-9 20.3-20.1 20.3z"/></svg> (source) packages - E.g. `~/Desktop/Packages/` or `/r/packages` and `/python/packages` <div class="note"> <p>There are manual approaches to create the initial package structure but we’ll use a tool to help automate this procedure. We’ll cover this within our working examples.</p> </div> --- # Enable version control .scrollable90[ Most package development never resides solely on one operating system nor with only one developer. - [GitHub](https://github.com/), - [GitLab](https://gitlab.com/), - [Azure Repos](https://azure.microsoft.com/en-us/services/devops/repos/) .bold[Step 1]: connect our local git repository to a remote repository so we can push, pull, and merge updates we make to the package .bold[Step 2]: set up a proper branching method for our work - __master__ branch is the "production" release branch. - __develop__ branch is the main branch where the source code always reflects a state with the latest delivered development changes for the next release. - __supporting branches__ are the branches where we do the majority of our work. These are the branches where we create new fixtures, fix bugs, improve documentation. <div class="tip"> <p>If you are unfamiliar with git and branching then we recommend these tutorials to get started:</p> <ul> <li><a href="https://guides.github.com/introduction/flow/">Understanding the GitHub flow</a></li> <li><a href="https://nvie.com/posts/a-successful-git-branching-model/">A successful Git branching model</a></li> </ul> </div> ] --- # Virtual environments - Virtual environments create an isolated environment for our project - Each project can have its own dependencies, regardless of what dependencies every other project has - Creates better reliability during the development process. <div class="tip"> <p>.bold[It is a best practice to always do your development work in virtual environments.] To learn more about virtual environments we suggest starting with the following tutorials:</p> <ul> <li><a href="https://realpython.com/python-virtual-environments-a-primer/">Python Virtual Environments: A Primer</a></li> <li><a href="https://rstudio.github.io/renv/articles/renv.html">renv: Project Environments for R</a></li> </ul> </div> --- # Add new functionality .scrollable90[ When adding new functionality we recommend following a .bold[test-driven development] (TDD) approach 1. __Add a test__: In TDD, each new feature or bug fix begins with writing a test. This test defines a function, improvements of a function, or a case in which a function results in a bug. 2. __Run test(s) and make sure the new test fails__: Before writing any source code, run the new test to ensure that it fails. 3. __Write the code__: The next step is to write some code that causes the test to pass. The new code written at this stage is not perfect and may, for example, pass the test in an inelegant way. That is acceptable because it will be improved and honed in Step 5. 4. __Run tests__: If all test cases now pass, the programmer can be confident that the new code meets the test requirements, and does not break or degrade any existing features. If they do not, the new code must be adjusted until they do. 5. __Refactor code__: Once the source code passes the test, you should spend time refactoring and cleaning it up. A growing code base can quickly get out of control. This step ensures you take the time to reduce [tech debt](https://en.wikipedia.org/wiki/Technical_debt). 6. __Repeat__: Starting with another new test, the cycle is then repeated to push forward the functionality of the source code. <div class="tip"> <p>If you are interested in learning more about the TDD philosophy we recommend the following books:</p> <ul> <li><a href="https://www.amazon.com/Pragmatic-Programmer-journey-mastery-Anniversary/dp/0135957052/ref=dp_ob_title_bk">The Pragramatic Programmer</a> (Ch. 41)</li> <li><a href="https://www.amazon.com/Clean-Code-Handbook-Software-Craftsmanship/dp/0132350882/ref=pd_bxgy_img_2/134-3713013-8748161?_encoding=UTF8&pd_rd_i=0132350882&pd_rd_r=c66a8eb8-13a3-4246-bf5c-47bff29f3be6&pd_rd_w=m2kCB&pd_rd_wg=NqeJv&pf_rd_p=4e3f7fc3-00c8-46a6-a4db-8457e6319578&pf_rd_r=R46745MCZ1C0V1ZD3QS4&psc=1&refRID=R46745MCZ1C0V1ZD3QS4">Clean Code</a> (Ch. 9)</li> <li><a href="https://www.amazon.com/Test-Driven-Development-Kent-Beck/dp/0321146530">Test Driven Development</a></li> </ul> </div> ] --- # Document Once your new functionality is successfully passing tests and has been refactored, you should make sure any necessary documentation is added or updated: - function specific documentation, - module level documentation, - any code in example notebooks or vignettes, - package website --- # Version control Lastly, we need to now save our work to our remote repository. This typically includes: - staging, - commiting and - pushing our changes. - Depending on the stage of our work we may be ready to do a pull request into the main development branch. -- <br><br> .center.bold.red[Seems like a lot of steps to keep track of. Often, this feels more convoluted on paper than when you are actually implementing so let's work through some examples!] --- class: inverse, hide-logo <br><br><br> [.center.white.font300[ <svg style="height:0.8em;top:.04em;position:relative;fill:white;" viewBox="0 0 581 512"><path d="M581 226.6C581 119.1 450.9 32 290.5 32S0 119.1 0 226.6C0 322.4 103.3 402 239.4 418.1V480h99.1v-61.5c24.3-2.7 47.6-7.4 69.4-13.9L448 480h112l-67.4-113.7c54.5-35.4 88.4-84.9 88.4-139.7zm-466.8 14.5c0-73.5 98.9-133 220.8-133s211.9 40.7 211.9 133c0 50.1-26.5 85-70.3 106.4-2.4-1.6-4.7-2.9-6.4-3.7-10.2-5.2-27.8-10.5-27.8-10.5s86.6-6.4 86.6-92.7-90.6-87.9-90.6-87.9h-199V361c-74.1-21.5-125.2-67.1-125.2-119.9zm225.1 38.3v-55.6c57.8 0 87.8-6.8 87.8 27.3 0 36.5-38.2 28.3-87.8 28.3zm-.9 72.5H365c10.8 0 18.9 11.7 24 19.2-16.1 1.9-33 2.8-50.6 2.9v-22.1z"/></svg> example ]](https://misk-data-science.github.io/misk-packages/notebooks/03-workflow.html#_workflow_example) --- class: inverse, hide-logo <br><br><br> [.center.white.font300[ <svg style="height:0.8em;top:.04em;position:relative;fill:white;" viewBox="0 0 448 512"><path d="M439.8 200.5c-7.7-30.9-22.3-54.2-53.4-54.2h-40.1v47.4c0 36.8-31.2 67.8-66.8 67.8H172.7c-29.2 0-53.4 25-53.4 54.3v101.8c0 29 25.2 46 53.4 54.3 33.8 9.9 66.3 11.7 106.8 0 26.9-7.8 53.4-23.5 53.4-54.3v-40.7H226.2v-13.6h160.2c31.1 0 42.6-21.7 53.4-54.2 11.2-33.5 10.7-65.7 0-108.6zM286.2 404c11.1 0 20.1 9.1 20.1 20.3 0 11.3-9 20.4-20.1 20.4-11 0-20.1-9.2-20.1-20.4.1-11.3 9.1-20.3 20.1-20.3zM167.8 248.1h106.8c29.7 0 53.4-24.5 53.4-54.3V91.9c0-29-24.4-50.7-53.4-55.6-35.8-5.9-74.7-5.6-106.8.1-45.2 8-53.4 24.7-53.4 55.6v40.7h106.9v13.6h-147c-31.1 0-58.3 18.7-66.8 54.2-9.8 40.7-10.2 66.1 0 108.6 7.6 31.6 25.7 54.2 56.8 54.2H101v-48.8c0-35.3 30.5-66.4 66.8-66.4zm-6.7-142.6c-11.1 0-20.1-9.1-20.1-20.3.1-11.3 9-20.4 20.1-20.4 11 0 20.1 9.2 20.1 20.4s-9 20.3-20.1 20.3z"/></svg> example ]](https://misk-data-science.github.io/misk-packages/notebooks/03-workflow.html#_workflow_example5) --- # Exercises <br> Pick at least one exercise for R and one for Python to complete. Write up a summary of the findings and share with a colleague: * R blended learning exercises 1. Read [Navigating the R Package Universe](https://journal.r-project.org/archive/2018/RJ-2018-058/RJ-2018-058.pdf). 2. Watch David Robinson's screencast on [creating an R data package](https://www.youtube.com/watch?edufilter=NULL&v=F4oUJp76KUY). * Python blended learning exercises 1. Read [Hitchhiker's Guide to Python: Packaging Your Code](https://docs.python-guide.org/shipping/packaging/). 2. Read [Python's New Package Landscape](http://andrewsforge.com/article/python-new-package-landscape/). --- class: misk-section-slide <br><br><br><br><br><br><br> .bold.font250[Package Metadata] --- # Metadata considerations .pull-left[ - Name & package description - Author & contact info - Dependencies - Versioning - License - Project source - Other ] .pull-right[ ] --- # Metadata considerations .pull-left[ - .bold[Name & package description] - Author & contact info - Dependencies - Versioning - License - Project source - Other ] .pull-right[ * Name of the package, * A shorter, one line description of the package provides, * A longer, multi-line description of the package. This is the elevator pitch of the key things that your package does and how important it is to others. ] --- # Metadata considerations .pull-left[ - Name & package description - .bold[Author & contact info] - Dependencies - Versioning - License - Project source - Other ] .pull-right[ - Minimum: primary author and email contact info - Optionally provide additional authors and contributors along with their contact info - Can even specify corporation relationships if necessary ] --- # Metadata considerations .pull-left[ - Name & package description - Author & contact info - .bold[Dependencies] - Versioning - License - Project source - Other ] .pull-right[ - Language version dependency - Required package dependencies - Suggested package dependencies - Developer dependencies ] --- # Metadata considerations .pull-left[ - Name & package description - Author & contact info - Dependencies - .bold[Versioning] - License - Project source - Other ] .pull-right[ We advise using a [Semantic Versioning](https://semver.org/) approach `<MAJOR>.<MINOR>.<PATCH>` (i.e. 0.9.1, 1.4.0) - `<MAJOR>` version when you make incompatible API changes. This signals that your updates will likely effect many users and will cause breaking changes to users' prior code. - `<MINOR>` version when you add functionality in a backwards compatible manner. This often includes adding a new feature that does not effect the existing code base. - `<PATCH>` version when you make backwards compatible bug fixes. ] --- # Metadata considerations .scrollable90[ .pull-left[ - Name & package description - Author & contact info - Dependencies - Versioning - .bold[License] - Project source - Other ] .pull-right[ The License field can be either a standard abbreviation for an open source license, like GPL-2 or BSD, or a pointer to a file containing more information, file `LICENSE`. - [MIT](https://tldrlegal.com/license/mit-license): This is a simple and permissive license. It lets people use and freely distribute your code subject to only one restriction: the license must always be distributed with the code. - [GPL-2](https://tldrlegal.com/license/gnu-general-public-license-v2) or [GPL-3](https://tldrlegal.com/license/gnu-general-public-license-v3-(gpl-3)): These are “copy-left” licenses. This means that anyone who distributes your code in a bundle must license the whole bundle in a GPL-compatible way. Additionally, anyone who distributes modified versions of your code (derivative works) must also make the source code available. GPL-3 is a little stricter than GPL-2, closing some older loopholes. - [CC0](https://tldrlegal.com/license/creative-commons-cc0-1.0-universal): It relinquishes all your rights on the code and data so that it can be freely used by anyone for any purpose. This is sometimes called putting it in the public domain, a term which is neither well-defined nor meaningful in all countries. <div class="tip"> <p>If you’d like to learn more about other common licenses, Github’s <a href="http://choosealicense.com/licenses/">choosealicense.com</a> is a good place to start. Another good resource is <a href="https://tldrlegal.com/" class="uri">https://tldrlegal.com/</a>, which explains the most important parts of each license.</p> </div> ]] --- # Metadata considerations .pull-left[ - Name & package description - Author & contact info - Dependencies - Versioning - License - .bold[Project source] - Other ] .pull-right[ It is common to include URLs to direct users to: - Where the source code resides (i.e. Github repo) - Where to post bugs, feature requests, etc. (i.e. Github issues) We may also want to point people to a package documentation website, the changelog, or other URLs that host important package information. ] --- # Metadata considerations .pull-left[ - Name & package description - Author & contact info - Dependencies - Versioning - License - Project source - .bold[Other] ] .pull-right[ As you'll see in the next couple sections, there are additional types of metadata that we can include such as deploying data or other files (i.e. LICENSE) with you package. However, the items listed in the previous section are the primary forms of metadata that we want to ensure we include. ] --- class: inverse, hide-logo <br><br><br> [.center.white.font300[ <svg style="height:0.8em;top:.04em;position:relative;fill:white;" viewBox="0 0 581 512"><path d="M581 226.6C581 119.1 450.9 32 290.5 32S0 119.1 0 226.6C0 322.4 103.3 402 239.4 418.1V480h99.1v-61.5c24.3-2.7 47.6-7.4 69.4-13.9L448 480h112l-67.4-113.7c54.5-35.4 88.4-84.9 88.4-139.7zm-466.8 14.5c0-73.5 98.9-133 220.8-133s211.9 40.7 211.9 133c0 50.1-26.5 85-70.3 106.4-2.4-1.6-4.7-2.9-6.4-3.7-10.2-5.2-27.8-10.5-27.8-10.5s86.6-6.4 86.6-92.7-90.6-87.9-90.6-87.9h-199V361c-74.1-21.5-125.2-67.1-125.2-119.9zm225.1 38.3v-55.6c57.8 0 87.8-6.8 87.8 27.3 0 36.5-38.2 28.3-87.8 28.3zm-.9 72.5H365c10.8 0 18.9 11.7 24 19.2-16.1 1.9-33 2.8-50.6 2.9v-22.1z"/></svg> example ]](https://misk-data-science.github.io/misk-packages/notebooks/05-metadata.html#_package_metadata_example) --- class: inverse, hide-logo <br><br><br> [.center.white.font300[ <svg style="height:0.8em;top:.04em;position:relative;fill:white;" viewBox="0 0 448 512"><path d="M439.8 200.5c-7.7-30.9-22.3-54.2-53.4-54.2h-40.1v47.4c0 36.8-31.2 67.8-66.8 67.8H172.7c-29.2 0-53.4 25-53.4 54.3v101.8c0 29 25.2 46 53.4 54.3 33.8 9.9 66.3 11.7 106.8 0 26.9-7.8 53.4-23.5 53.4-54.3v-40.7H226.2v-13.6h160.2c31.1 0 42.6-21.7 53.4-54.2 11.2-33.5 10.7-65.7 0-108.6zM286.2 404c11.1 0 20.1 9.1 20.1 20.3 0 11.3-9 20.4-20.1 20.4-11 0-20.1-9.2-20.1-20.4.1-11.3 9.1-20.3 20.1-20.3zM167.8 248.1h106.8c29.7 0 53.4-24.5 53.4-54.3V91.9c0-29-24.4-50.7-53.4-55.6-35.8-5.9-74.7-5.6-106.8.1-45.2 8-53.4 24.7-53.4 55.6v40.7h106.9v13.6h-147c-31.1 0-58.3 18.7-66.8 54.2-9.8 40.7-10.2 66.1 0 108.6 7.6 31.6 25.7 54.2 56.8 54.2H101v-48.8c0-35.3 30.5-66.4 66.8-66.4zm-6.7-142.6c-11.1 0-20.1-9.1-20.1-20.3.1-11.3 9-20.4 20.1-20.4 11 0 20.1 9.2 20.1 20.4s-9 20.3-20.1 20.3z"/></svg> example ]](https://misk-data-science.github.io/misk-packages/notebooks/05-metadata.html#_package_metadata_example9) --- # Exercises .scrollable90[ * Check out the metadata for the [dplyr](https://github.com/tidyverse/dplyr) package. * What is the current version number? * Who was the original creator? * What kind of license does it have? * What R version does it require? * What kind of dependencies are glue, rlang, DBI and lobstr? * Check out the metadata for the [pre-commit](https://github.com/pre-commit/pre-commit) package. Note that this package uses setup.cfg to hold its metadata. * What is the current version number? * Who is the author? * What kind of license does it have? * What Oython version does it require? * Name 4 package dependencies of pre-commit? * Building on to the initial R and Python packages you created in the portfolio building exercise: * Check out the LICENSE file in your package. * R package - Fill out the `Description` field for your package. - Add __dplyr__ as a required package dependency. - Add __purrr__ as a suggested package dependency. - Run `devtools::check()` to make sure the package builds successfully. * Python package - Create a `long_description` field for your package. - Add __numpy__ as a required package dependency. - Add __black__ as a developer suggested package dependency. - Activate your virtual environment, install your package in editable mode, ensure all developer dependencies are installed and rerun your test(s). ] --- class: misk-section-slide <br><br><br><br><br><br><br> .bold.font250[Source Code] --- # Organization Organizing your source code provides many benefits such as: - making the intentions of the code obvious, - making it easy for others to contribute, - making it easy for you to maintain, - making it easy to debug, - making it easy to expand. <div class="note"> <p>There are very few strict requirements in how you organize your code but what follows are general best practices.</p> </div> --- # Individual units .pull-left[ Individual units of functionality - should be <font size="-2">small.</font>, - should not require significant scrolling in your editor, - should be very clear regarding the intention and functionality they are providing, - should only _"do one thing"_. <div class="tip"> <p>One way to know if a function is doing more than one thing is if you can extract another function from it with a name that is not merely a restatement of its implementation.</p> </div> ] -- .pull-right[ Two main ways our code expands: 1. breadth: we add more functionality but not necessarily building onto existing functionality, 2. aggregation: we continue building onto existing functionality to create layers of abstraction. ] --- # Expanding code breadth * Expanding code breadth is easier to maintain. For our example package that currently has a `my_mean()` function, this could include adding new functions such as `my_median()` and `my_mode()`. * When expanding code breadth the main thing you should consider is where to put the new functionality. * Grouping like functionality into one file is good. For example, since `my_mean()`, `my_median()` and `my_mode()` are all forms of central tendencies we may put them in the same file while functions with other purposes (i.e. deviations) may go into a different file. <div class="tip"> <p>While you’re free to arrange functions into files as you wish, the two extremes are bad: don’t put all functions into one file and don’t put each function into its own separate file.</p> </div> --- # Aggregating functionality .pull-left[ * Often, we write code that builds on top of each other. * This is known as creating higher levels of abstraction. * For example, say we wanted to create a function that computes the z-score, which uses the mean and standard deviations. This is a higher level of abstraction and we should write our code in such a way that: 1. our code reads like a top-down narrative, 2. functionality is abstracted away and grouped at similar levels of abstraction. ] .pull-right[ ```python # highest abstracted level def z_score(): pass # next layer of abstractoin def my_mean(): pass def my_sd(): pass # lowest level of abstraction def validate_input(): pass ``` ] --- # Naming is important Naming is important but as our source code grows it becomes even moreso. Here are some good naming tips: - If a file contains a single function, give the file the same name as the function. - If a file contains multiple related functions, give it a concise, but descriptive name. For example, we could group `my_mean()`, `my_median()` and `my_mode()` together into a file named `central_tendencies`. - Sometimes you have many helper functions in your source code that all support one primary user facing API functionality. Put the user-facing API function/class into a file named `main` or `main_api`. - Many times people create a general `utils` file to hold general utility functions. This is not advised. Always try to find a proper home and name for all supporting functions. This is a [great read](https://breadcrumbscollector.tech/stop-naming-your-python-modules-utils/) explaining why. - Deprecated functions should live in a file or directory that makes it obvious they are deprecated (i.e. prefix file name with `deprec-`). --- # External vs internal When developing packages, we often build functions with two main purposes: 1. External use: functions that are designed to be used by the end-user. This is the primary purpose of writing a package, to make some functionality easier for an end-user. 2. Internal use: functions that are designed to help you, the developer, write efficient and effect code. Internal functions are usually not intended for external use by end-users. -- <div class="tip"> <p>Within both R and Python package you have the ability to make your functions be either externally or internally focused. In the examples that follow we will illustrate how but it is important that you apply the same rules to both external and internal functions - document and test <strong>all</strong> your functions. Although not a requirement this will make your life and other developer’s lives that want to contribute much easier.</p> </div> --- class: inverse, hide-logo <br><br><br> [.center.white.font300[ <svg style="height:0.8em;top:.04em;position:relative;fill:white;" viewBox="0 0 581 512"><path d="M581 226.6C581 119.1 450.9 32 290.5 32S0 119.1 0 226.6C0 322.4 103.3 402 239.4 418.1V480h99.1v-61.5c24.3-2.7 47.6-7.4 69.4-13.9L448 480h112l-67.4-113.7c54.5-35.4 88.4-84.9 88.4-139.7zm-466.8 14.5c0-73.5 98.9-133 220.8-133s211.9 40.7 211.9 133c0 50.1-26.5 85-70.3 106.4-2.4-1.6-4.7-2.9-6.4-3.7-10.2-5.2-27.8-10.5-27.8-10.5s86.6-6.4 86.6-92.7-90.6-87.9-90.6-87.9h-199V361c-74.1-21.5-125.2-67.1-125.2-119.9zm225.1 38.3v-55.6c57.8 0 87.8-6.8 87.8 27.3 0 36.5-38.2 28.3-87.8 28.3zm-.9 72.5H365c10.8 0 18.9 11.7 24 19.2-16.1 1.9-33 2.8-50.6 2.9v-22.1z"/></svg> example ]](https://misk-data-science.github.io/misk-packages/notebooks/06-source-code.html#_source_code_example) --- class: inverse, hide-logo <br><br><br> [.center.white.font300[ <svg style="height:0.8em;top:.04em;position:relative;fill:white;" viewBox="0 0 448 512"><path d="M439.8 200.5c-7.7-30.9-22.3-54.2-53.4-54.2h-40.1v47.4c0 36.8-31.2 67.8-66.8 67.8H172.7c-29.2 0-53.4 25-53.4 54.3v101.8c0 29 25.2 46 53.4 54.3 33.8 9.9 66.3 11.7 106.8 0 26.9-7.8 53.4-23.5 53.4-54.3v-40.7H226.2v-13.6h160.2c31.1 0 42.6-21.7 53.4-54.2 11.2-33.5 10.7-65.7 0-108.6zM286.2 404c11.1 0 20.1 9.1 20.1 20.3 0 11.3-9 20.4-20.1 20.4-11 0-20.1-9.2-20.1-20.4.1-11.3 9.1-20.3 20.1-20.3zM167.8 248.1h106.8c29.7 0 53.4-24.5 53.4-54.3V91.9c0-29-24.4-50.7-53.4-55.6-35.8-5.9-74.7-5.6-106.8.1-45.2 8-53.4 24.7-53.4 55.6v40.7h106.9v13.6h-147c-31.1 0-58.3 18.7-66.8 54.2-9.8 40.7-10.2 66.1 0 108.6 7.6 31.6 25.7 54.2 56.8 54.2H101v-48.8c0-35.3 30.5-66.4 66.8-66.4zm-6.7-142.6c-11.1 0-20.1-9.1-20.1-20.3.1-11.3 9-20.4 20.1-20.4 11 0 20.1 9.2 20.1 20.4s-9 20.3-20.1 20.3z"/></svg> example ]](https://misk-data-science.github.io/misk-packages/notebooks/06-source-code.html#_source_code_example3) --- # Exercises <br> 1. With the R and Python packages you created in the portfolio builder, work through the [R](#rexample) and [Python](#pyexample) examples above to add new functionality to your package. 2. Fork one of your classmates packages and: - create a new branch, - add a new function (and associated unit test) that is unique to the existing functions, - add a new function (and associated unit test) that builds onto one of their existing functions, - create a pull request to add this new functionality to your classmates develop branch. --- class: misk-section-slide <br><br><br><br><br><br><br> .bold.font250[Tests] --- # Why test Formalizing and automating the test structure and process helps with: .scrollable90[ * __Fewer bugs__. Because you’re explicit about how your code should behave you will have fewer bugs. The reason why is a bit like the reason double entry book-keeping works: because you describe the behaviour of your code in two places, both in your code and in your tests, you are able to check one against the other. By following this approach to testing, you can be sure that bugs that you’ve fixed in the past will never come back to haunt you. * __Better code structure__. Code that’s easy to test is usually better designed. This is because writing tests forces you to break up complicated parts of your code into separate functions that can work in isolation. This reduces duplication in your code. As a result, functions will be easier to test, understand and work with (it’ll be easier to combine them in new ways). * __Easier restarts__. If you always finish a coding session by creating a failing test (e.g. for the next feature you want to implement), testing makes it easier for you to pick up where you left off: your tests will let you know what to do next. * __Robust code__. If you know that all the major functionality of your package has an associated test, you can confidently make big changes without worrying about accidentally breaking something. For me, this is particularly useful when I think I have a simpler way to accomplish a task (usually the reason my solution is simpler is that I’ve forgotten an important use case!). ] --- # What to test .scrollable90[ * __Focus on testing the external interface to your functions__ - if you test the internal interface, then it’s harder to change the implementation in the future because as well as modifying the code, you’ll also need to update all the tests. * Strive to __test each behaviour in one and only one test__. Then if that behavior later changes you only need to update a single test. * You do not need to test _all_ of your code; however, we often __strive to test 85-95% of our code__. Focus your time on code that you’re not sure about, is fragile, or has complicated interdependencies. That said, we often find we make the most mistakes when we falsely assume that the problem is simple and doesn’t need any tests. So there is definitely a case for striving for 100% coverage. * __Always write a test when you discover a bug or want to create a feature enhancement__. Start by writing the tests, and then write the code that makes them pass. This reflects an important problem solving strategy: start by establishing your success criteria, how you know if you’ve solved the problem. * You should not only write tests that outline the successful expected outcome of a function but you should also __write tests to verify that expected messages, warnings, and errors are produced__. ] --- # Test organization * tests should live in a `tests/` directory at the root of the package * test files for both languages should start with `test` (i.e. `test-validation.R`, `test_summary_stats.py`) * Tests are organized hierarchically: expectations and assertions are grouped into tests which are organized in files: * An __expectation__ or __assertion__ is the atom of testing. It describes the expected result of a computation: Does it have the right value and right class? Does it produce error messages when it should? An expectation automates visual checking of results in the console. * A __test__ groups together one or more expectations/assertions to test the output from a simple function, a range of possibilities for a single parameter from a more complicated function, or tightly related functionality from across multiple functions. This is why they are sometimes called unit as they test one unit of functionality. * A __test file__ groups together multiple related tests. How you organize tests within files is your discretion; however, one best practice suggests that for every source code script there is a companion, similarly named test script (i.e. `summary_stats.py` & `test_summary_stats.py`). --- # Types of tests Many types of tests; however, the most common ones you should be thinking of initially are: .scrollable90[ * __unit tests__: - test one unit of functionality - focus on the output from a single function - foundational building block functions of your package - should be fast so you can run them often and, preferablly not require any special environment to run them (i.e. a Spark session) * __integration tests__: - tests that focus on processes and/or components that combine functionality from across multiple functions - ensure that all the lower abstraction level functions work nicely with one another when combined <div class="note"> <p>Often, integrated functionality becomes very large and sometimes can be slower to compute so it is not unusual for integration tests to be much slower than unit tests. However, we also do not need to run them as often as unit tests.</p> </div> ] --- # Writing tests .pull-left[ __Given-When-Then__ is a style of representing tests that can help guide you in writing your tests, make your tests more focused and better documented. * The __given__ part describes the state of the world before you begin the behavior you're specifying in this scenario. You can think of it as the pre-conditions to the test. * The __when__ section is the behavior that you're specifying, or the specific functionality you are applying to the given state of the world. * Finally the __then__ section describes the changes you expect due to the specified behavior. ] -- .pull-right[ A simple example is with our `my_mean()` function. In both languages we follow a common procedure: 1. __GIVEN__ a vector or list of integers from 0-10, 2. __WHEN__ we compute the mean value, 3. __THEN__ the result should be 5 ] --- # Additional resources - Build your knowledge of general software testing philosophies: - [The Pragramatic Programmer](https://www.amazon.com/Pragmatic-Programmer-journey-mastery-Anniversary/dp/0135957052/ref=dp_ob_title_bk) (Ch. 41) - [Clean Code](https://www.amazon.com/Clean-Code-Handbook-Software-Craftsmanship/dp/0132350882/ref=pd_bxgy_img_2/134-3713013-8748161?_encoding=UTF8&pd_rd_i=0132350882&pd_rd_r=c66a8eb8-13a3-4246-bf5c-47bff29f3be6&pd_rd_w=m2kCB&pd_rd_wg=NqeJv&pf_rd_p=4e3f7fc3-00c8-46a6-a4db-8457e6319578&pf_rd_r=R46745MCZ1C0V1ZD3QS4&psc=1&refRID=R46745MCZ1C0V1ZD3QS4) (Ch. 9) - [Test Driven Development](https://www.amazon.com/Test-Driven-Development-Kent-Beck/dp/0321146530) - Learn more about implementing software testing in <svg style="height:0.8em;top:.04em;position:relative;" viewBox="0 0 581 512"><path d="M581 226.6C581 119.1 450.9 32 290.5 32S0 119.1 0 226.6C0 322.4 103.3 402 239.4 418.1V480h99.1v-61.5c24.3-2.7 47.6-7.4 69.4-13.9L448 480h112l-67.4-113.7c54.5-35.4 88.4-84.9 88.4-139.7zm-466.8 14.5c0-73.5 98.9-133 220.8-133s211.9 40.7 211.9 133c0 50.1-26.5 85-70.3 106.4-2.4-1.6-4.7-2.9-6.4-3.7-10.2-5.2-27.8-10.5-27.8-10.5s86.6-6.4 86.6-92.7-90.6-87.9-90.6-87.9h-199V361c-74.1-21.5-125.2-67.1-125.2-119.9zm225.1 38.3v-55.6c57.8 0 87.8-6.8 87.8 27.3 0 36.5-38.2 28.3-87.8 28.3zm-.9 72.5H365c10.8 0 18.9 11.7 24 19.2-16.1 1.9-33 2.8-50.6 2.9v-22.1z"/></svg> and <svg style="height:0.8em;top:.04em;position:relative;" viewBox="0 0 448 512"><path d="M439.8 200.5c-7.7-30.9-22.3-54.2-53.4-54.2h-40.1v47.4c0 36.8-31.2 67.8-66.8 67.8H172.7c-29.2 0-53.4 25-53.4 54.3v101.8c0 29 25.2 46 53.4 54.3 33.8 9.9 66.3 11.7 106.8 0 26.9-7.8 53.4-23.5 53.4-54.3v-40.7H226.2v-13.6h160.2c31.1 0 42.6-21.7 53.4-54.2 11.2-33.5 10.7-65.7 0-108.6zM286.2 404c11.1 0 20.1 9.1 20.1 20.3 0 11.3-9 20.4-20.1 20.4-11 0-20.1-9.2-20.1-20.4.1-11.3 9.1-20.3 20.1-20.3zM167.8 248.1h106.8c29.7 0 53.4-24.5 53.4-54.3V91.9c0-29-24.4-50.7-53.4-55.6-35.8-5.9-74.7-5.6-106.8.1-45.2 8-53.4 24.7-53.4 55.6v40.7h106.9v13.6h-147c-31.1 0-58.3 18.7-66.8 54.2-9.8 40.7-10.2 66.1 0 108.6 7.6 31.6 25.7 54.2 56.8 54.2H101v-48.8c0-35.3 30.5-66.4 66.8-66.4zm-6.7-142.6c-11.1 0-20.1-9.1-20.1-20.3.1-11.3 9-20.4 20.1-20.4 11 0 20.1 9.2 20.1 20.4s-9 20.3-20.1 20.3z"/></svg>: - [Testing R Code](https://www.amazon.com/Testing-Code-Chapman-Hall-CRC/dp/1498763650) - [testthat documentation](https://testthat.r-lib.org/) - [Python Testing with pytest](https://www.amazon.com/Python-Testing-pytest-Effective-Scalable/dp/1680502409/ref=sr_1_3?keywords=Python+Testing+with+pytest&qid=1578174634&s=books&sr=1-3) - [Multiply your Testing Effectiveness with Parameterized Testing](https://us.pycon.org/2020/schedule/presentation/172/) - [pytest documentation](https://docs.pytest.org/en/latest/) --- class: inverse, hide-logo <br><br><br> [.center.white.font300[ <svg style="height:0.8em;top:.04em;position:relative;fill:white;" viewBox="0 0 581 512"><path d="M581 226.6C581 119.1 450.9 32 290.5 32S0 119.1 0 226.6C0 322.4 103.3 402 239.4 418.1V480h99.1v-61.5c24.3-2.7 47.6-7.4 69.4-13.9L448 480h112l-67.4-113.7c54.5-35.4 88.4-84.9 88.4-139.7zm-466.8 14.5c0-73.5 98.9-133 220.8-133s211.9 40.7 211.9 133c0 50.1-26.5 85-70.3 106.4-2.4-1.6-4.7-2.9-6.4-3.7-10.2-5.2-27.8-10.5-27.8-10.5s86.6-6.4 86.6-92.7-90.6-87.9-90.6-87.9h-199V361c-74.1-21.5-125.2-67.1-125.2-119.9zm225.1 38.3v-55.6c57.8 0 87.8-6.8 87.8 27.3 0 36.5-38.2 28.3-87.8 28.3zm-.9 72.5H365c10.8 0 18.9 11.7 24 19.2-16.1 1.9-33 2.8-50.6 2.9v-22.1z"/></svg> example ]](https://misk-data-science.github.io/misk-packages/notebooks/08-testing.html#_testing_example) --- class: inverse, hide-logo <br><br><br> [.center.white.font300[ <svg style="height:0.8em;top:.04em;position:relative;fill:white;" viewBox="0 0 448 512"><path d="M439.8 200.5c-7.7-30.9-22.3-54.2-53.4-54.2h-40.1v47.4c0 36.8-31.2 67.8-66.8 67.8H172.7c-29.2 0-53.4 25-53.4 54.3v101.8c0 29 25.2 46 53.4 54.3 33.8 9.9 66.3 11.7 106.8 0 26.9-7.8 53.4-23.5 53.4-54.3v-40.7H226.2v-13.6h160.2c31.1 0 42.6-21.7 53.4-54.2 11.2-33.5 10.7-65.7 0-108.6zM286.2 404c11.1 0 20.1 9.1 20.1 20.3 0 11.3-9 20.4-20.1 20.4-11 0-20.1-9.2-20.1-20.4.1-11.3 9.1-20.3 20.1-20.3zM167.8 248.1h106.8c29.7 0 53.4-24.5 53.4-54.3V91.9c0-29-24.4-50.7-53.4-55.6-35.8-5.9-74.7-5.6-106.8.1-45.2 8-53.4 24.7-53.4 55.6v40.7h106.9v13.6h-147c-31.1 0-58.3 18.7-66.8 54.2-9.8 40.7-10.2 66.1 0 108.6 7.6 31.6 25.7 54.2 56.8 54.2H101v-48.8c0-35.3 30.5-66.4 66.8-66.4zm-6.7-142.6c-11.1 0-20.1-9.1-20.1-20.3.1-11.3 9-20.4 20.1-20.4 11 0 20.1 9.2 20.1 20.4s-9 20.3-20.1 20.3z"/></svg> example ]](https://misk-data-science.github.io/misk-packages/notebooks/08-testing.html#_testing_example7) --- # Exercises <br> 1. With the R and Python packages you created in the portfolio builder, work through the [R](#rexample) and [Python](#pyexample) examples above to update your package's unit tests. 2. Fork one of your classmates packages and: - create a new branch, - review their unit tests, identify and add a new unit test, - add a new function (and associated unit test) that is unique to the existing functions. 3. Review the testing setup for the [usethis](https://github.com/r-lib/usethis) package. Identify something unique about this setup compared to the testing structure we outlined above. 4. Review the testing setup for the [scikit-learn](https://github.com/scikit-learn/scikit-learn/tree/master/sklearn/tests) package. Identify at least three tests where the documentation could be clearer by incorporating our __Given-When-Then__ approach. How would you write this improved documentation? --- class: misk-section-slide <br><br><br><br><br><br><br> .bold.font250[Object documentation] --- # Levels of documentation Multiple forms of documentation: - __Package-level documentation__: Top level description for what your package does and can point users to additional resources if necessary. - __Module-level documentation__: More common in <svg style="height:0.8em;top:.04em;position:relative;" viewBox="0 0 448 512"><path d="M439.8 200.5c-7.7-30.9-22.3-54.2-53.4-54.2h-40.1v47.4c0 36.8-31.2 67.8-66.8 67.8H172.7c-29.2 0-53.4 25-53.4 54.3v101.8c0 29 25.2 46 53.4 54.3 33.8 9.9 66.3 11.7 106.8 0 26.9-7.8 53.4-23.5 53.4-54.3v-40.7H226.2v-13.6h160.2c31.1 0 42.6-21.7 53.4-54.2 11.2-33.5 10.7-65.7 0-108.6zM286.2 404c11.1 0 20.1 9.1 20.1 20.3 0 11.3-9 20.4-20.1 20.4-11 0-20.1-9.2-20.1-20.4.1-11.3 9.1-20.3 20.1-20.3zM167.8 248.1h106.8c29.7 0 53.4-24.5 53.4-54.3V91.9c0-29-24.4-50.7-53.4-55.6-35.8-5.9-74.7-5.6-106.8.1-45.2 8-53.4 24.7-53.4 55.6v40.7h106.9v13.6h-147c-31.1 0-58.3 18.7-66.8 54.2-9.8 40.7-10.2 66.1 0 108.6 7.6 31.6 25.7 54.2 56.8 54.2H101v-48.8c0-35.3 30.5-66.4 66.8-66.4zm-6.7-142.6c-11.1 0-20.1-9.1-20.1-20.3.1-11.3 9-20.4 20.1-20.4 11 0 20.1 9.2 20.1 20.4s-9 20.3-20.1 20.3z"/></svg> than <svg style="height:0.8em;top:.04em;position:relative;" viewBox="0 0 581 512"><path d="M581 226.6C581 119.1 450.9 32 290.5 32S0 119.1 0 226.6C0 322.4 103.3 402 239.4 418.1V480h99.1v-61.5c24.3-2.7 47.6-7.4 69.4-13.9L448 480h112l-67.4-113.7c54.5-35.4 88.4-84.9 88.4-139.7zm-466.8 14.5c0-73.5 98.9-133 220.8-133s211.9 40.7 211.9 133c0 50.1-26.5 85-70.3 106.4-2.4-1.6-4.7-2.9-6.4-3.7-10.2-5.2-27.8-10.5-27.8-10.5s86.6-6.4 86.6-92.7-90.6-87.9-90.6-87.9h-199V361c-74.1-21.5-125.2-67.1-125.2-119.9zm225.1 38.3v-55.6c57.8 0 87.8-6.8 87.8 27.3 0 36.5-38.2 28.3-87.8 28.3zm-.9 72.5H365c10.8 0 18.9 11.7 24 19.2-16.1 1.9-33 2.8-50.6 2.9v-22.1z"/></svg> and helps to describe the purpose and contents of an individual module within a package. - __Function-level documentation__: The function documentation that you have already been exposed to in earlier modules. <br> <div class="tip"> <p>Documentation is one of the most important aspects of a good package</p> </div> --- class: inverse, hide-logo <br><br><br> [.center.white.font300[ <svg style="height:0.8em;top:.04em;position:relative;fill:white;" viewBox="0 0 581 512"><path d="M581 226.6C581 119.1 450.9 32 290.5 32S0 119.1 0 226.6C0 322.4 103.3 402 239.4 418.1V480h99.1v-61.5c24.3-2.7 47.6-7.4 69.4-13.9L448 480h112l-67.4-113.7c54.5-35.4 88.4-84.9 88.4-139.7zm-466.8 14.5c0-73.5 98.9-133 220.8-133s211.9 40.7 211.9 133c0 50.1-26.5 85-70.3 106.4-2.4-1.6-4.7-2.9-6.4-3.7-10.2-5.2-27.8-10.5-27.8-10.5s86.6-6.4 86.6-92.7-90.6-87.9-90.6-87.9h-199V361c-74.1-21.5-125.2-67.1-125.2-119.9zm225.1 38.3v-55.6c57.8 0 87.8-6.8 87.8 27.3 0 36.5-38.2 28.3-87.8 28.3zm-.9 72.5H365c10.8 0 18.9 11.7 24 19.2-16.1 1.9-33 2.8-50.6 2.9v-22.1z"/></svg> example ]](https://misk-data-science.github.io/misk-packages/notebooks/09-object-docs.html#_object_documentation_example) --- class: inverse, hide-logo <br><br><br> [.center.white.font300[ <svg style="height:0.8em;top:.04em;position:relative;fill:white;" viewBox="0 0 448 512"><path d="M439.8 200.5c-7.7-30.9-22.3-54.2-53.4-54.2h-40.1v47.4c0 36.8-31.2 67.8-66.8 67.8H172.7c-29.2 0-53.4 25-53.4 54.3v101.8c0 29 25.2 46 53.4 54.3 33.8 9.9 66.3 11.7 106.8 0 26.9-7.8 53.4-23.5 53.4-54.3v-40.7H226.2v-13.6h160.2c31.1 0 42.6-21.7 53.4-54.2 11.2-33.5 10.7-65.7 0-108.6zM286.2 404c11.1 0 20.1 9.1 20.1 20.3 0 11.3-9 20.4-20.1 20.4-11 0-20.1-9.2-20.1-20.4.1-11.3 9.1-20.3 20.1-20.3zM167.8 248.1h106.8c29.7 0 53.4-24.5 53.4-54.3V91.9c0-29-24.4-50.7-53.4-55.6-35.8-5.9-74.7-5.6-106.8.1-45.2 8-53.4 24.7-53.4 55.6v40.7h106.9v13.6h-147c-31.1 0-58.3 18.7-66.8 54.2-9.8 40.7-10.2 66.1 0 108.6 7.6 31.6 25.7 54.2 56.8 54.2H101v-48.8c0-35.3 30.5-66.4 66.8-66.4zm-6.7-142.6c-11.1 0-20.1-9.1-20.1-20.3.1-11.3 9-20.4 20.1-20.4 11 0 20.1 9.2 20.1 20.4s-9 20.3-20.1 20.3z"/></svg> example ]](https://misk-data-science.github.io/misk-packages/notebooks/09-object-docs.html#_object_documentation_example2) --- # Exercises Focusing on the R and Python packages you created in the portfolio builder: 1. Fork one of your classmates packages, create a new branch, add package-level documentation and create a pull request. 2. Fork another classmates packages, create a new branch, add module-level documentation and create a pull request. 3. Fork another classmates packages, create a new branch, add function-level documentation and create a pull request. 4. __Optional__: Review the referenced documentation for [type-hints](#typehints) add type hints to your existing code. Be sure to run [mypy](https://mypy.readthedocs.io/en/stable/) on your package to ensure your type hints are implemented correctly. <div class="note"> <p>As a reviewer of the above pull requests, be sure to thoroughly review the contributors changes. It is ok to ask for edits or corrections on a pull request. And as a submitter of the above pull requests, if the maintainer requests edits or even rejects your pull request, don’t take this personally. Listen to their reasons why, learn from them and move on.</p> </div> --- class: misk-section-slide <br><br><br><br><br><br><br> .bold.font250[Changelog] --- # Changelog .pull-left[ * A curated, chronologically ordered list of notable changes for each version of a project * README --> new users * Changelog --> existing users * Gives you a detailed view into the development process * Makes it easier for users and contributors to see precisely what notable changes have been made between each release (or version) of the package ] .pull-right[ ![](figures/example-changelog.png) ] --- # How to log .pull-left[ .bold.center[What to log] * Release version and date of release * New features __added__ * __Changes__ in existing functionality * __Deprecated__ functionality (features/functionality that soon-to-be removed) * __Removed__ functionality * Bug __fixes__ ] -- .pull-right[ .bold.center[What not to do] * Commit log diffs * Ignoring deprecations ] --- # How to log * Typically contained in a file named `CHANGELOG`, `NEWS`, `RELEASES`, or `HISTORY` (.md or .rst) * Release version & date should be displayed (reverse chronological order) * Common to keep an `Unreleased` section at the top to track upcoming changes until you have determined the appropriate version bump * Common headings to capture types of changes: - `Added` - `Changed` - `Deprecated` - `Removed` - `Fixed` - `Refactor` * Related to an issue in GitHub, include the issue number in parentheses, e.g. (#10). If an item is related to a pull request, include the pull request number and the author, e.g. (#101, \@bradleyboehmke). --- # Example: markdown .scrollable90[ .code70[ ```md # Changelog All notable changes to this project will be documented in this file. ## Unreleased ### Added - Feature `xxx`: allows you to do some great stuff. <Provide good summary and you can even supply examples>. - Feature `xxx`: allows you do this other great stuff. ### Removed - Feature `xxx`: version 1.0.0 deprecated feature `xxx` has been removed. - Param `xxx`: version 1.0.0 deprecated param `xxx` has been removed. ### Fixed - Bug in `xxx`: was throwing `xxxx` error when doing <provide good summary> (#87). - Bug in `xxx`: was returning a integer instead of a float when <summary> (#86). ## Version 1.0.0 ### Added - Feature `xxx`: allows you to ... <summary>. - Param `xxx`: added to feature `xyz` to allow for... <summary>. ### Deprecated - Feature `xxx`: deprecated in favor of feature `xxx` (#98, @johndoe). - Param `xxx`: deprecated in favor of param `xxx` (#97, @janedoe). ## Version 0.9.1 ### Fixed - Bug in xxx: <summary> (#74). - Bug in xxx: <summary> (#73). - Bug in xxx: <summary> (#72). ## Version ... ... ``` ]] --- # Example: output .scrollable90.font90[ ## Changelog All notable changes to this project will be documented in this file. ### Unreleased #### Added - Feature `xxx`: allows you to do some great stuff. `<Provide good summary and you can even supply examples>`. - Feature `xxx`: allows you do this other great stuff. #### Removed - Feature `xxx`: version 1.0.0 deprecated feature `xxx` has been removed. - Param `xxx`: version 1.0.0 deprecated param `xxx` has been removed. #### Fixed - Bug in `xxx`: was throwing `xxxx` error when doing `<provide good summary>` (#87). - Bug in `xxx`: was returning a integer instead of a float when `<summary>` (#86). ### Version 1.0.0 #### Added - Feature `xxx`: allows you to ... `<summary>`. - Param `xxx`: added to feature `xyz` to allow for... `<summary>`. #### Deprecated - Feature `xxx`: deprecated in favor of feature `xxx` (#98, @johndoe). - Param `xxx`: deprecated in favor of param `xxx` (#97, @janedoe). ### Version 0.9.1 #### Fixed - Bug in xxx: `<summary>` (#74). - Bug in xxx: `<summary>` (#73). - Bug in xxx: `<summary>` (#72). ## Version ... ... ] --- # Exercises <br> 1. The [Pandas](https://github.com/pandas-dev/pandas) package is known for very detailed release notes. You can read them [here](https://pandas.pydata.org/docs/whatsnew/index.html) and see the source code [here](https://github.com/pandas-dev/pandas/tree/master/doc/source/whatsnew). What are 3 unique attributes about Pandas' approach to documenting changes? 2. Go to your <svg style="height:0.8em;top:.04em;position:relative;" viewBox="0 0 581 512"><path d="M581 226.6C581 119.1 450.9 32 290.5 32S0 119.1 0 226.6C0 322.4 103.3 402 239.4 418.1V480h99.1v-61.5c24.3-2.7 47.6-7.4 69.4-13.9L448 480h112l-67.4-113.7c54.5-35.4 88.4-84.9 88.4-139.7zm-466.8 14.5c0-73.5 98.9-133 220.8-133s211.9 40.7 211.9 133c0 50.1-26.5 85-70.3 106.4-2.4-1.6-4.7-2.9-6.4-3.7-10.2-5.2-27.8-10.5-27.8-10.5s86.6-6.4 86.6-92.7-90.6-87.9-90.6-87.9h-199V361c-74.1-21.5-125.2-67.1-125.2-119.9zm225.1 38.3v-55.6c57.8 0 87.8-6.8 87.8 27.3 0 36.5-38.2 28.3-87.8 28.3zm-.9 72.5H365c10.8 0 18.9 11.7 24 19.2-16.1 1.9-33 2.8-50.6 2.9v-22.1z"/></svg> and/or <svg style="height:0.8em;top:.04em;position:relative;" viewBox="0 0 448 512"><path d="M439.8 200.5c-7.7-30.9-22.3-54.2-53.4-54.2h-40.1v47.4c0 36.8-31.2 67.8-66.8 67.8H172.7c-29.2 0-53.4 25-53.4 54.3v101.8c0 29 25.2 46 53.4 54.3 33.8 9.9 66.3 11.7 106.8 0 26.9-7.8 53.4-23.5 53.4-54.3v-40.7H226.2v-13.6h160.2c31.1 0 42.6-21.7 53.4-54.2 11.2-33.5 10.7-65.7 0-108.6zM286.2 404c11.1 0 20.1 9.1 20.1 20.3 0 11.3-9 20.4-20.1 20.4-11 0-20.1-9.2-20.1-20.4.1-11.3 9.1-20.3 20.1-20.3zM167.8 248.1h106.8c29.7 0 53.4-24.5 53.4-54.3V91.9c0-29-24.4-50.7-53.4-55.6-35.8-5.9-74.7-5.6-106.8.1-45.2 8-53.4 24.7-53.4 55.6v40.7h106.9v13.6h-147c-31.1 0-58.3 18.7-66.8 54.2-9.8 40.7-10.2 66.1 0 108.6 7.6 31.6 25.7 54.2 56.8 54.2H101v-48.8c0-35.3 30.5-66.4 66.8-66.4zm-6.7-142.6c-11.1 0-20.1-9.1-20.1-20.3.1-11.3 9-20.4 20.1-20.4 11 0 20.1 9.2 20.1 20.4s-9 20.3-20.1 20.3z"/></svg> package and check out the existing changelog. 3. Add appropriate sections within the current version you are working to capture the features __added__ and any other changes you have made. --- class: misk-section-slide <br><br><br><br><br><br><br> .bold.font250[README] --- # A newcomers introduction * A [README](https://en.wikipedia.org/wiki/README) is a text file (often `.md` or `.rst`) that introduces and explains a project * Should always be in the top level directory of the package directory * Often the first document a user or develop will encounter when first learning about your package - understand what your package is about, - how to install it, - how to start using it, - how to communicate with the developers/maintainers. -- <div class="tip"> <p>A great README quickly gains trust from users and other developers!</p> </div> --- # Structure A simple way to structure your README is as follows: 1. A paragraph that describes the high-level purpose of the package. 2. Installation instructions, giving code that can be copied and pasted into an end-user's editor. 3. An example that shows how to use the package to solve a simple problem along with where the user can find more detailed examples if they exist. 4. How a user can report problems, bugs, or ideas for enhancements. 5. Where to find details about contributing to the package (optional). --- # Badges .scrollable90[ Common to see badges at the top of the README such as: - ![](https://img.shields.io/badge/version-1.0.0-blue.svg) - ![](https://img.shields.io/badge/lifecycle-experimental-orange.svg) - ![](https://img.shields.io/badge/coverage-100%25-brightgreen.svg) - ![](https://img.shields.io/badge/build-passing-brightgreen.svg) Not required but often add a level of confidence and professionalism to a package. Most informative and common include: - [Version number](https://shields.io/category/version) signaling the current version available. - [Lifecycle](https://www.tidyverse.org/lifecycle/) of your package to convey the maturity of your package. - [Build status](https://shields.io/category/build) to show that the current version builds successfully. - [Code coverage](https://shields.io/category/coverage) to show how much of your source code is tested. <div class="tip"> <p>Here are some places where you can access/reference common badges you can use in your packages:</p> <ul> <li><a href="https://shields.io/" class="uri">https://shields.io/</a></li> <li><a href="https://naereen.github.io/badges/" class="uri">https://naereen.github.io/badges/</a></li> <li><a href="https://www.tidyverse.org/lifecycle/" class="uri">https://www.tidyverse.org/lifecycle/</a></li> </ul> </div> ] --- # Great examples A simple README can go a long way. Here are some great resources to learn more about producing great READMEs and also some fantastic examples: - https://www.makeareadme.com/ - https://github.com/matiassingers/awesome-readme <div class="todo"> <p>Take 15 minutes to check out the above resources.</p> </div> --- # Great examples A simple README can go a long way. Here are some great resources to learn more about producing great READMEs and also some fantastic examples: - https://www.makeareadme.com/ - https://github.com/matiassingers/awesome-readme <div class="todo"> <p>Take 15 minutes to check out the above resources. .bold.red[Thoughts?]</p> </div> --- # <svg style="height:0.8em;top:.04em;position:relative;fill:white;" viewBox="0 0 581 512"><path d="M581 226.6C581 119.1 450.9 32 290.5 32S0 119.1 0 226.6C0 322.4 103.3 402 239.4 418.1V480h99.1v-61.5c24.3-2.7 47.6-7.4 69.4-13.9L448 480h112l-67.4-113.7c54.5-35.4 88.4-84.9 88.4-139.7zm-466.8 14.5c0-73.5 98.9-133 220.8-133s211.9 40.7 211.9 133c0 50.1-26.5 85-70.3 106.4-2.4-1.6-4.7-2.9-6.4-3.7-10.2-5.2-27.8-10.5-27.8-10.5s86.6-6.4 86.6-92.7-90.6-87.9-90.6-87.9h-199V361c-74.1-21.5-125.2-67.1-125.2-119.9zm225.1 38.3v-55.6c57.8 0 87.8-6.8 87.8 27.3 0 36.5-38.2 28.3-87.8 28.3zm-.9 72.5H365c10.8 0 18.9 11.7 24 19.2-16.1 1.9-33 2.8-50.6 2.9v-22.1z"/></svg> example .pull-left[ ![](figures/README_raw.png) ] .pull-right[ ![](figures/README.png) ] --- # <svg style="height:0.8em;top:.04em;position:relative;fill:white;" viewBox="0 0 448 512"><path d="M439.8 200.5c-7.7-30.9-22.3-54.2-53.4-54.2h-40.1v47.4c0 36.8-31.2 67.8-66.8 67.8H172.7c-29.2 0-53.4 25-53.4 54.3v101.8c0 29 25.2 46 53.4 54.3 33.8 9.9 66.3 11.7 106.8 0 26.9-7.8 53.4-23.5 53.4-54.3v-40.7H226.2v-13.6h160.2c31.1 0 42.6-21.7 53.4-54.2 11.2-33.5 10.7-65.7 0-108.6zM286.2 404c11.1 0 20.1 9.1 20.1 20.3 0 11.3-9 20.4-20.1 20.4-11 0-20.1-9.2-20.1-20.4.1-11.3 9.1-20.3 20.1-20.3zM167.8 248.1h106.8c29.7 0 53.4-24.5 53.4-54.3V91.9c0-29-24.4-50.7-53.4-55.6-35.8-5.9-74.7-5.6-106.8.1-45.2 8-53.4 24.7-53.4 55.6v40.7h106.9v13.6h-147c-31.1 0-58.3 18.7-66.8 54.2-9.8 40.7-10.2 66.1 0 108.6 7.6 31.6 25.7 54.2 56.8 54.2H101v-48.8c0-35.3 30.5-66.4 66.8-66.4zm-6.7-142.6c-11.1 0-20.1-9.1-20.1-20.3.1-11.3 9-20.4 20.1-20.4 11 0 20.1 9.2 20.1 20.4s-9 20.3-20.1 20.3z"/></svg> example .pull-left[ ![](figures/README_raw_Py.png) ] .pull-right[ ![](figures/README_Py.png) ] --- # Exercises 1. Review the READMEs for these packages: - [mypy](https://github.com/python/mypy) - [parsnip](https://github.com/tidymodels/parsnip) - [black](https://github.com/psf/black) - [skimr](https://github.com/ropensci/skimr) 2. Identify 3 things in the above packages' READMEs that you think is very helpful as a newcomer to that package. 3. Identify 3 things in the above packages' READMEs that you think could be improved to help newcomers to that package. 4. Using what you've learned improve upon your package's README. Be sure to do this in a feature branch and perform a pull request to the develop but have at least 2 other classmates review your pull request. --- class: misk-section-slide <br><br><br><br><br><br><br> .bold.font250[Long-form Documentation] --- # Why? * The README is designed to get a new user up and running with your package as quickly as possible. * However, you should not go into too much depth in the README because it can be overwhelming. * As your package grows and becomes more complex, you will likely want to create tutorials that are designed to give the user the next step of instruction with greater detailed, __longer-form documentation__. --- # Vignettes & notebook tutorials * <svg style="height:0.8em;top:.04em;position:relative;" viewBox="0 0 581 512"><path d="M581 226.6C581 119.1 450.9 32 290.5 32S0 119.1 0 226.6C0 322.4 103.3 402 239.4 418.1V480h99.1v-61.5c24.3-2.7 47.6-7.4 69.4-13.9L448 480h112l-67.4-113.7c54.5-35.4 88.4-84.9 88.4-139.7zm-466.8 14.5c0-73.5 98.9-133 220.8-133s211.9 40.7 211.9 133c0 50.1-26.5 85-70.3 106.4-2.4-1.6-4.7-2.9-6.4-3.7-10.2-5.2-27.8-10.5-27.8-10.5s86.6-6.4 86.6-92.7-90.6-87.9-90.6-87.9h-199V361c-74.1-21.5-125.2-67.1-125.2-119.9zm225.1 38.3v-55.6c57.8 0 87.8-6.8 87.8 27.3 0 36.5-38.2 28.3-87.8 28.3zm-.9 72.5H365c10.8 0 18.9 11.7 24 19.2-16.1 1.9-33 2.8-50.6 2.9v-22.1z"/></svg>: commonly called _"vignettes"_ * <svg style="height:0.8em;top:.04em;position:relative;" viewBox="0 0 581 512"><path d="M581 226.6C581 119.1 450.9 32 290.5 32S0 119.1 0 226.6C0 322.4 103.3 402 239.4 418.1V480h99.1v-61.5c24.3-2.7 47.6-7.4 69.4-13.9L448 480h112l-67.4-113.7c54.5-35.4 88.4-84.9 88.4-139.7zm-466.8 14.5c0-73.5 98.9-133 220.8-133s211.9 40.7 211.9 133c0 50.1-26.5 85-70.3 106.4-2.4-1.6-4.7-2.9-6.4-3.7-10.2-5.2-27.8-10.5-27.8-10.5s86.6-6.4 86.6-92.7-90.6-87.9-90.6-87.9h-199V361c-74.1-21.5-125.2-67.1-125.2-119.9zm225.1 38.3v-55.6c57.8 0 87.8-6.8 87.8 27.3 0 36.5-38.2 28.3-87.8 28.3zm-.9 72.5H365c10.8 0 18.9 11.7 24 19.2-16.1 1.9-33 2.8-50.6 2.9v-22.1z"/></svg>: common to store these tutorials in Jupyter notebooks <div class="tip"> <p>Need to put yourself in the readers’ shoes, and adopt a “beginner’s mind”</p> </div> --- # Vignettes & notebook tutorials * <svg style="height:0.8em;top:.04em;position:relative;" viewBox="0 0 581 512"><path d="M581 226.6C581 119.1 450.9 32 290.5 32S0 119.1 0 226.6C0 322.4 103.3 402 239.4 418.1V480h99.1v-61.5c24.3-2.7 47.6-7.4 69.4-13.9L448 480h112l-67.4-113.7c54.5-35.4 88.4-84.9 88.4-139.7zm-466.8 14.5c0-73.5 98.9-133 220.8-133s211.9 40.7 211.9 133c0 50.1-26.5 85-70.3 106.4-2.4-1.6-4.7-2.9-6.4-3.7-10.2-5.2-27.8-10.5-27.8-10.5s86.6-6.4 86.6-92.7-90.6-87.9-90.6-87.9h-199V361c-74.1-21.5-125.2-67.1-125.2-119.9zm225.1 38.3v-55.6c57.8 0 87.8-6.8 87.8 27.3 0 36.5-38.2 28.3-87.8 28.3zm-.9 72.5H365c10.8 0 18.9 11.7 24 19.2-16.1 1.9-33 2.8-50.6 2.9v-22.1z"/></svg>: commonly called _"vignettes"_ * <svg style="height:0.8em;top:.04em;position:relative;" viewBox="0 0 581 512"><path d="M581 226.6C581 119.1 450.9 32 290.5 32S0 119.1 0 226.6C0 322.4 103.3 402 239.4 418.1V480h99.1v-61.5c24.3-2.7 47.6-7.4 69.4-13.9L448 480h112l-67.4-113.7c54.5-35.4 88.4-84.9 88.4-139.7zm-466.8 14.5c0-73.5 98.9-133 220.8-133s211.9 40.7 211.9 133c0 50.1-26.5 85-70.3 106.4-2.4-1.6-4.7-2.9-6.4-3.7-10.2-5.2-27.8-10.5-27.8-10.5s86.6-6.4 86.6-92.7-90.6-87.9-90.6-87.9h-199V361c-74.1-21.5-125.2-67.1-125.2-119.9zm225.1 38.3v-55.6c57.8 0 87.8-6.8 87.8 27.3 0 36.5-38.2 28.3-87.8 28.3zm-.9 72.5H365c10.8 0 18.9 11.7 24 19.2-16.1 1.9-33 2.8-50.6 2.9v-22.1z"/></svg>: common to store these tutorials in Jupyter notebooks <div class="todo"> <p>Take 15 minutes to review these examples:</p> <ul> <li>R <ul> <li><a href="https://pkgdown.r-lib.org/articles/pkgdown.html" class="uri">https://pkgdown.r-lib.org/articles/pkgdown.html</a></li> <li><a href="https://dplyr.tidyverse.org/articles/dplyr.html" class="uri">https://dplyr.tidyverse.org/articles/dplyr.html</a></li> <li><a href="https://parsnip.tidymodels.org/articles/parsnip_Intro.html" class="uri">https://parsnip.tidymodels.org/articles/parsnip_Intro.html</a></li> </ul></li> <li>Python <ul> <li><a href="https://numpy.org/devdocs/user/quickstart.html" class="uri">https://numpy.org/devdocs/user/quickstart.html</a></li> <li><a href="https://github.com/has2k1/plotnine-examples/tree/master/plotnine_examples" class="uri">https://github.com/has2k1/plotnine-examples/tree/master/plotnine_examples</a></li> <li><a href="https://github.com/scikit-learn/scikit-learn/tree/master/examples" class="uri">https://github.com/scikit-learn/scikit-learn/tree/master/examples</a></li> </ul></li> </ul> </div> --- class: inverse, hide-logo <br><br><br> [.center.white.font300[ <svg style="height:0.8em;top:.04em;position:relative;fill:white;" viewBox="0 0 581 512"><path d="M581 226.6C581 119.1 450.9 32 290.5 32S0 119.1 0 226.6C0 322.4 103.3 402 239.4 418.1V480h99.1v-61.5c24.3-2.7 47.6-7.4 69.4-13.9L448 480h112l-67.4-113.7c54.5-35.4 88.4-84.9 88.4-139.7zm-466.8 14.5c0-73.5 98.9-133 220.8-133s211.9 40.7 211.9 133c0 50.1-26.5 85-70.3 106.4-2.4-1.6-4.7-2.9-6.4-3.7-10.2-5.2-27.8-10.5-27.8-10.5s86.6-6.4 86.6-92.7-90.6-87.9-90.6-87.9h-199V361c-74.1-21.5-125.2-67.1-125.2-119.9zm225.1 38.3v-55.6c57.8 0 87.8-6.8 87.8 27.3 0 36.5-38.2 28.3-87.8 28.3zm-.9 72.5H365c10.8 0 18.9 11.7 24 19.2-16.1 1.9-33 2.8-50.6 2.9v-22.1z"/></svg> example ]](https://misk-data-science.github.io/misk-packages/notebooks/12-long-form-docs.html#_tutorial_docs_example) --- class: inverse, hide-logo <br><br><br> [.center.white.font300[ <svg style="height:0.8em;top:.04em;position:relative;fill:white;" viewBox="0 0 448 512"><path d="M439.8 200.5c-7.7-30.9-22.3-54.2-53.4-54.2h-40.1v47.4c0 36.8-31.2 67.8-66.8 67.8H172.7c-29.2 0-53.4 25-53.4 54.3v101.8c0 29 25.2 46 53.4 54.3 33.8 9.9 66.3 11.7 106.8 0 26.9-7.8 53.4-23.5 53.4-54.3v-40.7H226.2v-13.6h160.2c31.1 0 42.6-21.7 53.4-54.2 11.2-33.5 10.7-65.7 0-108.6zM286.2 404c11.1 0 20.1 9.1 20.1 20.3 0 11.3-9 20.4-20.1 20.4-11 0-20.1-9.2-20.1-20.4.1-11.3 9.1-20.3 20.1-20.3zM167.8 248.1h106.8c29.7 0 53.4-24.5 53.4-54.3V91.9c0-29-24.4-50.7-53.4-55.6-35.8-5.9-74.7-5.6-106.8.1-45.2 8-53.4 24.7-53.4 55.6v40.7h106.9v13.6h-147c-31.1 0-58.3 18.7-66.8 54.2-9.8 40.7-10.2 66.1 0 108.6 7.6 31.6 25.7 54.2 56.8 54.2H101v-48.8c0-35.3 30.5-66.4 66.8-66.4zm-6.7-142.6c-11.1 0-20.1-9.1-20.1-20.3.1-11.3 9-20.4 20.1-20.4 11 0 20.1 9.2 20.1 20.4s-9 20.3-20.1 20.3z"/></svg> example ]](https://misk-data-science.github.io/misk-packages/notebooks/12-long-form-docs.html#_tutorial_docs_example2) --- # Exercises 1. Review these packages long-form documentation and identify three strengths that these all have in common and then identify particular weaknesses in one or more of them. * <svg style="height:0.8em;top:.04em;position:relative;" viewBox="0 0 581 512"><path d="M581 226.6C581 119.1 450.9 32 290.5 32S0 119.1 0 226.6C0 322.4 103.3 402 239.4 418.1V480h99.1v-61.5c24.3-2.7 47.6-7.4 69.4-13.9L448 480h112l-67.4-113.7c54.5-35.4 88.4-84.9 88.4-139.7zm-466.8 14.5c0-73.5 98.9-133 220.8-133s211.9 40.7 211.9 133c0 50.1-26.5 85-70.3 106.4-2.4-1.6-4.7-2.9-6.4-3.7-10.2-5.2-27.8-10.5-27.8-10.5s86.6-6.4 86.6-92.7-90.6-87.9-90.6-87.9h-199V361c-74.1-21.5-125.2-67.1-125.2-119.9zm225.1 38.3v-55.6c57.8 0 87.8-6.8 87.8 27.3 0 36.5-38.2 28.3-87.8 28.3zm-.9 72.5H365c10.8 0 18.9 11.7 24 19.2-16.1 1.9-33 2.8-50.6 2.9v-22.1z"/></svg> - https://pkgdown.r-lib.org/articles/pkgdown.html - https://dplyr.tidyverse.org/articles/dplyr.html - https://parsnip.tidymodels.org/articles/parsnip_Intro.html * <svg style="height:0.8em;top:.04em;position:relative;" viewBox="0 0 448 512"><path d="M439.8 200.5c-7.7-30.9-22.3-54.2-53.4-54.2h-40.1v47.4c0 36.8-31.2 67.8-66.8 67.8H172.7c-29.2 0-53.4 25-53.4 54.3v101.8c0 29 25.2 46 53.4 54.3 33.8 9.9 66.3 11.7 106.8 0 26.9-7.8 53.4-23.5 53.4-54.3v-40.7H226.2v-13.6h160.2c31.1 0 42.6-21.7 53.4-54.2 11.2-33.5 10.7-65.7 0-108.6zM286.2 404c11.1 0 20.1 9.1 20.1 20.3 0 11.3-9 20.4-20.1 20.4-11 0-20.1-9.2-20.1-20.4.1-11.3 9.1-20.3 20.1-20.3zM167.8 248.1h106.8c29.7 0 53.4-24.5 53.4-54.3V91.9c0-29-24.4-50.7-53.4-55.6-35.8-5.9-74.7-5.6-106.8.1-45.2 8-53.4 24.7-53.4 55.6v40.7h106.9v13.6h-147c-31.1 0-58.3 18.7-66.8 54.2-9.8 40.7-10.2 66.1 0 108.6 7.6 31.6 25.7 54.2 56.8 54.2H101v-48.8c0-35.3 30.5-66.4 66.8-66.4zm-6.7-142.6c-11.1 0-20.1-9.1-20.1-20.3.1-11.3 9-20.4 20.1-20.4 11 0 20.1 9.2 20.1 20.4s-9 20.3-20.1 20.3z"/></svg> - https://numpy.org/devdocs/user/quickstart.html - https://github.com/has2k1/plotnine-examples/tree/master/plotnine_examples - https://github.com/scikit-learn/scikit-learn/tree/master/examples 2. Add an introductory vignette to help newcomers use your R package. 3. Add an introductory vignette to help newcomers use your Python package. --- class: misk-section-slide <br><br><br><br><br><br><br> .bold.font250[Package Website] --- # Package website Both <svg style="height:0.8em;top:.04em;position:relative;" viewBox="0 0 448 512"><path d="M439.8 200.5c-7.7-30.9-22.3-54.2-53.4-54.2h-40.1v47.4c0 36.8-31.2 67.8-66.8 67.8H172.7c-29.2 0-53.4 25-53.4 54.3v101.8c0 29 25.2 46 53.4 54.3 33.8 9.9 66.3 11.7 106.8 0 26.9-7.8 53.4-23.5 53.4-54.3v-40.7H226.2v-13.6h160.2c31.1 0 42.6-21.7 53.4-54.2 11.2-33.5 10.7-65.7 0-108.6zM286.2 404c11.1 0 20.1 9.1 20.1 20.3 0 11.3-9 20.4-20.1 20.4-11 0-20.1-9.2-20.1-20.4.1-11.3 9.1-20.3 20.1-20.3zM167.8 248.1h106.8c29.7 0 53.4-24.5 53.4-54.3V91.9c0-29-24.4-50.7-53.4-55.6-35.8-5.9-74.7-5.6-106.8.1-45.2 8-53.4 24.7-53.4 55.6v40.7h106.9v13.6h-147c-31.1 0-58.3 18.7-66.8 54.2-9.8 40.7-10.2 66.1 0 108.6 7.6 31.6 25.7 54.2 56.8 54.2H101v-48.8c0-35.3 30.5-66.4 66.8-66.4zm-6.7-142.6c-11.1 0-20.1-9.1-20.1-20.3.1-11.3 9-20.4 20.1-20.4 11 0 20.1 9.2 20.1 20.4s-9 20.3-20.1 20.3z"/></svg> and <svg style="height:0.8em;top:.04em;position:relative;" viewBox="0 0 581 512"><path d="M581 226.6C581 119.1 450.9 32 290.5 32S0 119.1 0 226.6C0 322.4 103.3 402 239.4 418.1V480h99.1v-61.5c24.3-2.7 47.6-7.4 69.4-13.9L448 480h112l-67.4-113.7c54.5-35.4 88.4-84.9 88.4-139.7zm-466.8 14.5c0-73.5 98.9-133 220.8-133s211.9 40.7 211.9 133c0 50.1-26.5 85-70.3 106.4-2.4-1.6-4.7-2.9-6.4-3.7-10.2-5.2-27.8-10.5-27.8-10.5s86.6-6.4 86.6-92.7-90.6-87.9-90.6-87.9h-199V361c-74.1-21.5-125.2-67.1-125.2-119.9zm225.1 38.3v-55.6c57.8 0 87.8-6.8 87.8 27.3 0 36.5-38.2 28.3-87.8 28.3zm-.9 72.5H365c10.8 0 18.9 11.7 24 19.2-16.1 1.9-33 2.8-50.6 2.9v-22.1z"/></svg> have documentation generators that will allow you to set up a website for your package rather easily .pull-left[ .center[<svg style="height:50;" viewBox="0 0 581 512"><path d="M581 226.6C581 119.1 450.9 32 290.5 32S0 119.1 0 226.6C0 322.4 103.3 402 239.4 418.1V480h99.1v-61.5c24.3-2.7 47.6-7.4 69.4-13.9L448 480h112l-67.4-113.7c54.5-35.4 88.4-84.9 88.4-139.7zm-466.8 14.5c0-73.5 98.9-133 220.8-133s211.9 40.7 211.9 133c0 50.1-26.5 85-70.3 106.4-2.4-1.6-4.7-2.9-6.4-3.7-10.2-5.2-27.8-10.5-27.8-10.5s86.6-6.4 86.6-92.7-90.6-87.9-90.6-87.9h-199V361c-74.1-21.5-125.2-67.1-125.2-119.9zm225.1 38.3v-55.6c57.8 0 87.8-6.8 87.8 27.3 0 36.5-38.2 28.3-87.8 28.3zm-.9 72.5H365c10.8 0 18.9 11.7 24 19.2-16.1 1.9-33 2.8-50.6 2.9v-22.1z"/></svg>] ![](figures/pkgdown.png) ] .pull-right[ .center[<svg style="height:50;" viewBox="0 0 448 512"><path d="M439.8 200.5c-7.7-30.9-22.3-54.2-53.4-54.2h-40.1v47.4c0 36.8-31.2 67.8-66.8 67.8H172.7c-29.2 0-53.4 25-53.4 54.3v101.8c0 29 25.2 46 53.4 54.3 33.8 9.9 66.3 11.7 106.8 0 26.9-7.8 53.4-23.5 53.4-54.3v-40.7H226.2v-13.6h160.2c31.1 0 42.6-21.7 53.4-54.2 11.2-33.5 10.7-65.7 0-108.6zM286.2 404c11.1 0 20.1 9.1 20.1 20.3 0 11.3-9 20.4-20.1 20.4-11 0-20.1-9.2-20.1-20.4.1-11.3 9.1-20.3 20.1-20.3zM167.8 248.1h106.8c29.7 0 53.4-24.5 53.4-54.3V91.9c0-29-24.4-50.7-53.4-55.6-35.8-5.9-74.7-5.6-106.8.1-45.2 8-53.4 24.7-53.4 55.6v40.7h106.9v13.6h-147c-31.1 0-58.3 18.7-66.8 54.2-9.8 40.7-10.2 66.1 0 108.6 7.6 31.6 25.7 54.2 56.8 54.2H101v-48.8c0-35.3 30.5-66.4 66.8-66.4zm-6.7-142.6c-11.1 0-20.1-9.1-20.1-20.3.1-11.3 9-20.4 20.1-20.4 11 0 20.1 9.2 20.1 20.4s-9 20.3-20.1 20.3z"/></svg>] ![](figures/sphinx.png) ] --- # Package website Both <svg style="height:0.8em;top:.04em;position:relative;" viewBox="0 0 448 512"><path d="M439.8 200.5c-7.7-30.9-22.3-54.2-53.4-54.2h-40.1v47.4c0 36.8-31.2 67.8-66.8 67.8H172.7c-29.2 0-53.4 25-53.4 54.3v101.8c0 29 25.2 46 53.4 54.3 33.8 9.9 66.3 11.7 106.8 0 26.9-7.8 53.4-23.5 53.4-54.3v-40.7H226.2v-13.6h160.2c31.1 0 42.6-21.7 53.4-54.2 11.2-33.5 10.7-65.7 0-108.6zM286.2 404c11.1 0 20.1 9.1 20.1 20.3 0 11.3-9 20.4-20.1 20.4-11 0-20.1-9.2-20.1-20.4.1-11.3 9.1-20.3 20.1-20.3zM167.8 248.1h106.8c29.7 0 53.4-24.5 53.4-54.3V91.9c0-29-24.4-50.7-53.4-55.6-35.8-5.9-74.7-5.6-106.8.1-45.2 8-53.4 24.7-53.4 55.6v40.7h106.9v13.6h-147c-31.1 0-58.3 18.7-66.8 54.2-9.8 40.7-10.2 66.1 0 108.6 7.6 31.6 25.7 54.2 56.8 54.2H101v-48.8c0-35.3 30.5-66.4 66.8-66.4zm-6.7-142.6c-11.1 0-20.1-9.1-20.1-20.3.1-11.3 9-20.4 20.1-20.4 11 0 20.1 9.2 20.1 20.4s-9 20.3-20.1 20.3z"/></svg> and <svg style="height:0.8em;top:.04em;position:relative;" viewBox="0 0 581 512"><path d="M581 226.6C581 119.1 450.9 32 290.5 32S0 119.1 0 226.6C0 322.4 103.3 402 239.4 418.1V480h99.1v-61.5c24.3-2.7 47.6-7.4 69.4-13.9L448 480h112l-67.4-113.7c54.5-35.4 88.4-84.9 88.4-139.7zm-466.8 14.5c0-73.5 98.9-133 220.8-133s211.9 40.7 211.9 133c0 50.1-26.5 85-70.3 106.4-2.4-1.6-4.7-2.9-6.4-3.7-10.2-5.2-27.8-10.5-27.8-10.5s86.6-6.4 86.6-92.7-90.6-87.9-90.6-87.9h-199V361c-74.1-21.5-125.2-67.1-125.2-119.9zm225.1 38.3v-55.6c57.8 0 87.8-6.8 87.8 27.3 0 36.5-38.2 28.3-87.8 28.3zm-.9 72.5H365c10.8 0 18.9 11.7 24 19.2-16.1 1.9-33 2.8-50.6 2.9v-22.1z"/></svg> have documentation generators that will allow you to set up a website for your package rather easily .pull-left[ .center[<svg style="height:50;" viewBox="0 0 581 512"><path d="M581 226.6C581 119.1 450.9 32 290.5 32S0 119.1 0 226.6C0 322.4 103.3 402 239.4 418.1V480h99.1v-61.5c24.3-2.7 47.6-7.4 69.4-13.9L448 480h112l-67.4-113.7c54.5-35.4 88.4-84.9 88.4-139.7zm-466.8 14.5c0-73.5 98.9-133 220.8-133s211.9 40.7 211.9 133c0 50.1-26.5 85-70.3 106.4-2.4-1.6-4.7-2.9-6.4-3.7-10.2-5.2-27.8-10.5-27.8-10.5s86.6-6.4 86.6-92.7-90.6-87.9-90.6-87.9h-199V361c-74.1-21.5-125.2-67.1-125.2-119.9zm225.1 38.3v-55.6c57.8 0 87.8-6.8 87.8 27.3 0 36.5-38.2 28.3-87.8 28.3zm-.9 72.5H365c10.8 0 18.9 11.7 24 19.2-16.1 1.9-33 2.8-50.6 2.9v-22.1z"/></svg>] - [bayesplot](http://mc-stan.org/bayesplot/index.html) - [vip](https://koalaverse.github.io/vip/index.html) - [recipes](https://recipes.tidymodels.org/) ] .pull-right[ .center[<svg style="height:50;" viewBox="0 0 448 512"><path d="M439.8 200.5c-7.7-30.9-22.3-54.2-53.4-54.2h-40.1v47.4c0 36.8-31.2 67.8-66.8 67.8H172.7c-29.2 0-53.4 25-53.4 54.3v101.8c0 29 25.2 46 53.4 54.3 33.8 9.9 66.3 11.7 106.8 0 26.9-7.8 53.4-23.5 53.4-54.3v-40.7H226.2v-13.6h160.2c31.1 0 42.6-21.7 53.4-54.2 11.2-33.5 10.7-65.7 0-108.6zM286.2 404c11.1 0 20.1 9.1 20.1 20.3 0 11.3-9 20.4-20.1 20.4-11 0-20.1-9.2-20.1-20.4.1-11.3 9.1-20.3 20.1-20.3zM167.8 248.1h106.8c29.7 0 53.4-24.5 53.4-54.3V91.9c0-29-24.4-50.7-53.4-55.6-35.8-5.9-74.7-5.6-106.8.1-45.2 8-53.4 24.7-53.4 55.6v40.7h106.9v13.6h-147c-31.1 0-58.3 18.7-66.8 54.2-9.8 40.7-10.2 66.1 0 108.6 7.6 31.6 25.7 54.2 56.8 54.2H101v-48.8c0-35.3 30.5-66.4 66.8-66.4zm-6.7-142.6c-11.1 0-20.1-9.1-20.1-20.3.1-11.3 9-20.4 20.1-20.4 11 0 20.1 9.2 20.1 20.4s-9 20.3-20.1 20.3z"/></svg>] - [numpy](https://numpydoc.readthedocs.io/en/latest/) - [pandas](https://pandas.pydata.org/docs/) - [plotnine](https://plotnine.readthedocs.io/en/stable/) ] <div class="todo"> <p>Spend 15 minutes reviewing these great examples.</p> </div> --- # Github docs In both cases, the website documentation is contained in a `/docs` subdirectory of the package .pull-left[ .center.bold[Step 1] ![](figures/github-repo-settings.png) ] -- .pull-right[ .center.bold[Step 2] ![](figures/github-repo-enable-github-pages.png) ] --- # Github docs In both cases, the website documentation is contained in a `/docs` subdirectory of the package <div class="warning"> <p>This will generate a URL (i.e. <a href="https://bradleyboehmke.github.io/myfirstpkg/" class="uri">https://bradleyboehmke.github.io/myfirstpkg/</a>). Currently this URL will not render anything (actually, it will render a 404 <em>file not found</em> error) since we haven’t produced the website docs but once we do create our website files, we can then share this URL so people can view our website.</p> </div> <br> .center.bold.red[Let's go ahead and create our package websites!] --- class: inverse, hide-logo <br><br><br> [.center.white.font300[ <svg style="height:0.8em;top:.04em;position:relative;fill:white;" viewBox="0 0 581 512"><path d="M581 226.6C581 119.1 450.9 32 290.5 32S0 119.1 0 226.6C0 322.4 103.3 402 239.4 418.1V480h99.1v-61.5c24.3-2.7 47.6-7.4 69.4-13.9L448 480h112l-67.4-113.7c54.5-35.4 88.4-84.9 88.4-139.7zm-466.8 14.5c0-73.5 98.9-133 220.8-133s211.9 40.7 211.9 133c0 50.1-26.5 85-70.3 106.4-2.4-1.6-4.7-2.9-6.4-3.7-10.2-5.2-27.8-10.5-27.8-10.5s86.6-6.4 86.6-92.7-90.6-87.9-90.6-87.9h-199V361c-74.1-21.5-125.2-67.1-125.2-119.9zm225.1 38.3v-55.6c57.8 0 87.8-6.8 87.8 27.3 0 36.5-38.2 28.3-87.8 28.3zm-.9 72.5H365c10.8 0 18.9 11.7 24 19.2-16.1 1.9-33 2.8-50.6 2.9v-22.1z"/></svg> example ]](https://misk-data-science.github.io/misk-packages/notebooks/13-website.html#_website_example) --- class: inverse, hide-logo <br><br><br> [.center.white.font300[ <svg style="height:0.8em;top:.04em;position:relative;fill:white;" viewBox="0 0 448 512"><path d="M439.8 200.5c-7.7-30.9-22.3-54.2-53.4-54.2h-40.1v47.4c0 36.8-31.2 67.8-66.8 67.8H172.7c-29.2 0-53.4 25-53.4 54.3v101.8c0 29 25.2 46 53.4 54.3 33.8 9.9 66.3 11.7 106.8 0 26.9-7.8 53.4-23.5 53.4-54.3v-40.7H226.2v-13.6h160.2c31.1 0 42.6-21.7 53.4-54.2 11.2-33.5 10.7-65.7 0-108.6zM286.2 404c11.1 0 20.1 9.1 20.1 20.3 0 11.3-9 20.4-20.1 20.4-11 0-20.1-9.2-20.1-20.4.1-11.3 9.1-20.3 20.1-20.3zM167.8 248.1h106.8c29.7 0 53.4-24.5 53.4-54.3V91.9c0-29-24.4-50.7-53.4-55.6-35.8-5.9-74.7-5.6-106.8.1-45.2 8-53.4 24.7-53.4 55.6v40.7h106.9v13.6h-147c-31.1 0-58.3 18.7-66.8 54.2-9.8 40.7-10.2 66.1 0 108.6 7.6 31.6 25.7 54.2 56.8 54.2H101v-48.8c0-35.3 30.5-66.4 66.8-66.4zm-6.7-142.6c-11.1 0-20.1-9.1-20.1-20.3.1-11.3 9-20.4 20.1-20.4 11 0 20.1 9.2 20.1 20.4s-9 20.3-20.1 20.3z"/></svg> example ]](https://misk-data-science.github.io/misk-packages/notebooks/13-website.html#_website_example2) --- # Exercises 1. Review these packages websites. Can you identify the source code location for these websites? - Python packages - [numpy](https://numpydoc.readthedocs.io/en/latest/) - [pandas](https://pandas.pydata.org/docs/) - [plotnine](https://plotnine.readthedocs.io/en/stable/) - R packages - [bayesplot](http://mc-stan.org/bayesplot/index.html) - [vip](https://koalaverse.github.io/vip/index.html) - [recipes](https://recipes.tidymodels.org/) 2. Add a basic website for your R package. Now review this [vignette](https://pkgdown.r-lib.org/articles/pkgdown.html) for using and customizing pkgdown and make at least 3 changes to the format of your website. 3. Add a basic website for your Python package. Now review this [guide](https://www.sphinx-doc.org/en/master/usage/quickstart.html) for using and customizing Sphinx docs and make at least 3 changes to the format of your website. --- class: misk-section-slide <br><br><br><br><br><br><br> .bold.font250[Other Components] --- # Staying simple So far we have covered the important and common components of <svg style="height:0.8em;top:.04em;position:relative;" viewBox="0 0 581 512"><path d="M581 226.6C581 119.1 450.9 32 290.5 32S0 119.1 0 226.6C0 322.4 103.3 402 239.4 418.1V480h99.1v-61.5c24.3-2.7 47.6-7.4 69.4-13.9L448 480h112l-67.4-113.7c54.5-35.4 88.4-84.9 88.4-139.7zm-466.8 14.5c0-73.5 98.9-133 220.8-133s211.9 40.7 211.9 133c0 50.1-26.5 85-70.3 106.4-2.4-1.6-4.7-2.9-6.4-3.7-10.2-5.2-27.8-10.5-27.8-10.5s86.6-6.4 86.6-92.7-90.6-87.9-90.6-87.9h-199V361c-74.1-21.5-125.2-67.1-125.2-119.9zm225.1 38.3v-55.6c57.8 0 87.8-6.8 87.8 27.3 0 36.5-38.2 28.3-87.8 28.3zm-.9 72.5H365c10.8 0 18.9 11.7 24 19.2-16.1 1.9-33 2.8-50.6 2.9v-22.1z"/></svg> and <svg style="height:0.8em;top:.04em;position:relative;" viewBox="0 0 448 512"><path d="M439.8 200.5c-7.7-30.9-22.3-54.2-53.4-54.2h-40.1v47.4c0 36.8-31.2 67.8-66.8 67.8H172.7c-29.2 0-53.4 25-53.4 54.3v101.8c0 29 25.2 46 53.4 54.3 33.8 9.9 66.3 11.7 106.8 0 26.9-7.8 53.4-23.5 53.4-54.3v-40.7H226.2v-13.6h160.2c31.1 0 42.6-21.7 53.4-54.2 11.2-33.5 10.7-65.7 0-108.6zM286.2 404c11.1 0 20.1 9.1 20.1 20.3 0 11.3-9 20.4-20.1 20.4-11 0-20.1-9.2-20.1-20.4.1-11.3 9.1-20.3 20.1-20.3zM167.8 248.1h106.8c29.7 0 53.4-24.5 53.4-54.3V91.9c0-29-24.4-50.7-53.4-55.6-35.8-5.9-74.7-5.6-106.8.1-45.2 8-53.4 24.7-53.4 55.6v40.7h106.9v13.6h-147c-31.1 0-58.3 18.7-66.8 54.2-9.8 40.7-10.2 66.1 0 108.6 7.6 31.6 25.7 54.2 56.8 54.2H101v-48.8c0-35.3 30.5-66.4 66.8-66.4zm-6.7-142.6c-11.1 0-20.1-9.1-20.1-20.3.1-11.3 9-20.4 20.1-20.4 11 0 20.1 9.2 20.1 20.4s-9 20.3-20.1 20.3z"/></svg> packages that you should be concerned with. However, as you become more involved in package development in either of these languages, you will start to see additional files or components present in packaging projects. -- <br> <div class="tip"> <p>You will want to familiarize yourself with these other components as you do more packaging but they are not necessary to make packages. Learn the basics first, then work on expanding into these components!</p> </div> --- # Other <svg style="height:0.8em;top:.04em;position:relative;fill:white;" viewBox="0 0 581 512"><path d="M581 226.6C581 119.1 450.9 32 290.5 32S0 119.1 0 226.6C0 322.4 103.3 402 239.4 418.1V480h99.1v-61.5c24.3-2.7 47.6-7.4 69.4-13.9L448 480h112l-67.4-113.7c54.5-35.4 88.4-84.9 88.4-139.7zm-466.8 14.5c0-73.5 98.9-133 220.8-133s211.9 40.7 211.9 133c0 50.1-26.5 85-70.3 106.4-2.4-1.6-4.7-2.9-6.4-3.7-10.2-5.2-27.8-10.5-27.8-10.5s86.6-6.4 86.6-92.7-90.6-87.9-90.6-87.9h-199V361c-74.1-21.5-125.2-67.1-125.2-119.9zm225.1 38.3v-55.6c57.8 0 87.8-6.8 87.8 27.3 0 36.5-38.2 28.3-87.8 28.3zm-.9 72.5H365c10.8 0 18.9 11.7 24 19.2-16.1 1.9-33 2.8-50.6 2.9v-22.1z"/></svg> package components - `data/`: Store binary data and make it available to end-users. Read [here](https://r-pkgs.org/data.html) for more details. - `src/`: Incorporate C and C++ code to speed up your package functionality. Learn more about incorporating C and C++ into your packages [here](https://r-pkgs.org/src.html). - `.Rbuildignore`: Lists files that we need to have around but that should not be included when building the R package from source. Read more [here](https://r-pkgs.org/package-structure-state.html#rbuildignore). - `NAMESPACE`: Declares the functions your package exports for external use and the external functions your package imports from other packages. You will not edit this directly, rather, it will be updated from the roxygen documentation. Read more [here](https://r-pkgs.org/namespace.html#namespace). - `inst/`: Files/folders that should be copied and unmodified into the installed R package folder. Read more [here](https://r-pkgs.org/inst.html). - `cran-comments.md`: When submitting your package to CRAN, this file provides important comments that you want to provide to the reviewers of your package. Learn more [here](https://r-pkgs.org/release.html#release-process). --- # Other <svg style="height:0.8em;top:.04em;position:relative;fill:white;" viewBox="0 0 448 512"><path d="M439.8 200.5c-7.7-30.9-22.3-54.2-53.4-54.2h-40.1v47.4c0 36.8-31.2 67.8-66.8 67.8H172.7c-29.2 0-53.4 25-53.4 54.3v101.8c0 29 25.2 46 53.4 54.3 33.8 9.9 66.3 11.7 106.8 0 26.9-7.8 53.4-23.5 53.4-54.3v-40.7H226.2v-13.6h160.2c31.1 0 42.6-21.7 53.4-54.2 11.2-33.5 10.7-65.7 0-108.6zM286.2 404c11.1 0 20.1 9.1 20.1 20.3 0 11.3-9 20.4-20.1 20.4-11 0-20.1-9.2-20.1-20.4.1-11.3 9.1-20.3 20.1-20.3zM167.8 248.1h106.8c29.7 0 53.4-24.5 53.4-54.3V91.9c0-29-24.4-50.7-53.4-55.6-35.8-5.9-74.7-5.6-106.8.1-45.2 8-53.4 24.7-53.4 55.6v40.7h106.9v13.6h-147c-31.1 0-58.3 18.7-66.8 54.2-9.8 40.7-10.2 66.1 0 108.6 7.6 31.6 25.7 54.2 56.8 54.2H101v-48.8c0-35.3 30.5-66.4 66.8-66.4zm-6.7-142.6c-11.1 0-20.1-9.1-20.1-20.3.1-11.3 9-20.4 20.1-20.4 11 0 20.1 9.2 20.1 20.4s-9 20.3-20.1 20.3z"/></svg> package components .scrollable90[ - `setup.cfg`: A configuration file that allows you to specify certain configurations for various tooling. Read more [here](https://docs.python.org/3/distutils/configfile.html). - `requirements.txt`: Popular alternative to specifying developer dependencies. Learn more [here](https://medium.com/@boscacci/why-and-how-to-make-a-requirements-txt-f329c685181e#:~:text=In%20short%2C%20we%20generate%20and,Python%20code%20we've%20written.). - `<pkg_name>.toml`: Python's attempt to provide a single, common config file for packages to specify their build dependencies. You can read more [here](https://www.python.org/dev/peps/pep-0518/) and [here](https://snarky.ca/what-the-heck-is-pyproject-toml/). - `MANIFEST.ini`: Distribute additional files such as the CHANGELOG, AUTHOR, config files (`.yml`) used by your source code, or even distributing data files. Read more [here](https://packaging.python.org/guides/using-manifest-in/). - `pytest.ini`: pytest configuration file. Read more [here](https://docs.pytest.org/en/2.7.3/customize.html). - `mypy.ini`: [mypy](http://mypy-lang.org/) is an optional static type checker commonly used for Python projects. Similar to pytest, you can cusomtize the mypy configuration in a `mypy.ini` file or, as in our case, the `setup.cfg` file. - `.pre-commit-config.yaml`: An optional, yet fantastic, tool that automates the application of git hooks to your code. Git hooks allow you to automatically check that certain characteristics exist in your code before allowing you to `git add, commit, push` your code. Read more at https://pre-commit.com/. ] --- class: misk-section-slide <br><br><br><br><br><br><br> .bold.font250[Continuous Integration] --- # CI/CD <br> .font130[ - **Continuous integration (CI)** is the idea of continuously testing our Python package as we make changes to our source code - **Continuous delivery/deployment (CD)** is the idea that we continuously and automatically deploy our package to a production environment ] --- # Why <br> .font110[ - So far we relied on ourselves to manually run any tests and checks to ensure our code is working. - As our project gets larger and we have more contributors, it becomes important that we automate this process to ensure any changes made to our code base are done so with our quality expectations met. - Automating this process will guarantee that all changes to our code have been tested. Moreso, we can set it up so that it is obvious if a commit or pull request is passing or failing. <br> .center.bold.red[CI/CD helps ensure a certain level of quality exists] ] --- # Tools .pull-left.font120[ There are many tools that can do this. Some common public source tools include: - [Travis-CI](https://travis-ci.org/) - [Azure Pipelines](https://azure.microsoft.com/en-us/services/devops/pipelines/) - [GitHub Actions](https://github.com/features/actions) ] .pull-right[ <img src="figures/travis-ci-azure.jpg" width="75%" height="75%" style="display: block; margin: auto;" /> <img src="figures/github-actions.png" width="50%" height="50%" style="display: block; margin: auto;" /> ] <div class="note"> <p>All these services not only will automate the process of testing your package, but they can also automate the process of deploying your package to a production environment (or CRAN and PyPI).</p> </div> --- class: inverse, hide-logo <br><br><br> [.center.white.font300[ <svg style="height:0.8em;top:.04em;position:relative;fill:white;" viewBox="0 0 581 512"><path d="M581 226.6C581 119.1 450.9 32 290.5 32S0 119.1 0 226.6C0 322.4 103.3 402 239.4 418.1V480h99.1v-61.5c24.3-2.7 47.6-7.4 69.4-13.9L448 480h112l-67.4-113.7c54.5-35.4 88.4-84.9 88.4-139.7zm-466.8 14.5c0-73.5 98.9-133 220.8-133s211.9 40.7 211.9 133c0 50.1-26.5 85-70.3 106.4-2.4-1.6-4.7-2.9-6.4-3.7-10.2-5.2-27.8-10.5-27.8-10.5s86.6-6.4 86.6-92.7-90.6-87.9-90.6-87.9h-199V361c-74.1-21.5-125.2-67.1-125.2-119.9zm225.1 38.3v-55.6c57.8 0 87.8-6.8 87.8 27.3 0 36.5-38.2 28.3-87.8 28.3zm-.9 72.5H365c10.8 0 18.9 11.7 24 19.2-16.1 1.9-33 2.8-50.6 2.9v-22.1z"/></svg> example ]](https://misk-data-science.github.io/misk-packages/notebooks/16-ci.html#_ci_example) --- class: inverse, hide-logo <br><br><br> [.center.white.font300[ <svg style="height:0.8em;top:.04em;position:relative;fill:white;" viewBox="0 0 448 512"><path d="M439.8 200.5c-7.7-30.9-22.3-54.2-53.4-54.2h-40.1v47.4c0 36.8-31.2 67.8-66.8 67.8H172.7c-29.2 0-53.4 25-53.4 54.3v101.8c0 29 25.2 46 53.4 54.3 33.8 9.9 66.3 11.7 106.8 0 26.9-7.8 53.4-23.5 53.4-54.3v-40.7H226.2v-13.6h160.2c31.1 0 42.6-21.7 53.4-54.2 11.2-33.5 10.7-65.7 0-108.6zM286.2 404c11.1 0 20.1 9.1 20.1 20.3 0 11.3-9 20.4-20.1 20.4-11 0-20.1-9.2-20.1-20.4.1-11.3 9.1-20.3 20.1-20.3zM167.8 248.1h106.8c29.7 0 53.4-24.5 53.4-54.3V91.9c0-29-24.4-50.7-53.4-55.6-35.8-5.9-74.7-5.6-106.8.1-45.2 8-53.4 24.7-53.4 55.6v40.7h106.9v13.6h-147c-31.1 0-58.3 18.7-66.8 54.2-9.8 40.7-10.2 66.1 0 108.6 7.6 31.6 25.7 54.2 56.8 54.2H101v-48.8c0-35.3 30.5-66.4 66.8-66.4zm-6.7-142.6c-11.1 0-20.1-9.1-20.1-20.3.1-11.3 9-20.4 20.1-20.4 11 0 20.1 9.2 20.1 20.4s-9 20.3-20.1 20.3z"/></svg> example ]](https://misk-data-science.github.io/misk-packages/notebooks/16-ci.html#_ci_example2) --- # Exercises <br> 1. Add a github actions CI pipeline to your R and Python packages. 2. Add a Travis CI pipeline to your R and Python packages. 3. Add an Azure CI pipeline to your R and Python packages. 4. Identify and review the CI pipeline workflows in at least 3 different [Tidyverse packages](https://github.com/tidyverse). Implement at least one new CI pipeline workflow attribute you discover into your package pipeline. 5. Identify and review the CI pipeline workflows in at least 3 different [Anthony Sottile repositories](https://github.com/asottile?tab=repositories). Implement at least one new CI pipeline workflow attribute you discover into your package pipeline.