I don't think it's a huge leap to say your functions are stored in this .py file vs. in a cell above. the magic commands simply allow you to reload the functions automatically without needing to reimport / reload the kernel.
It is a huge gap when the notebook is documentation. Moving functions out either duplicates documentation or separates them. Notebooks are vastly superior for documentation. Extracting functions from notebooks into a py file isn't difficult (see nbdev for example)
Notebooks are good for demonstration, not documentation. They are bad for documentation because you need to put docstrings and comments in the code while desiring to put the same content in the markdown.
I typically don't find docstrings are particularly useful for documenting an analysis or understanding what a function is doing. Even analyses that are repeated with different data. When you are debugging or reviewing an analysis, the docstrings are the least of your concerns. Validity is far more important.
I would take a notebook that demonstrates what's happening to my data any day over a docstring that may or may not be correct. Particularly if I have to render it in yet a third system to juggle and manually keep in sync.
So we've gone from notebooks to... notebook + py + docstring... I'm sure we can think of another useless layer of indirection to bolt on.
If you are putting your docstrings in the py and they contain things like equations or figures or images, you have to render them to get the same effect that you would have just reviewing cells in the notebook. And as far as I know you can't even spread docstrings out over the body of a functions, the way you could if you broke the function up into cells. Docstrings are not literate programming.
I realize you think the problem is that I don't understand what docstrings are, but it's more that you don't understand the sorts of documentation I'm talking about. docstrings are fine for a certain type of limited documentation that is formatted to prevent syntax errors. But they are extremely limited. Reading docstring/doxygen style documentation is the slowest and most error prone in my experience. Particularly when you are describing a complex calculation.
> If your logic is broken up across cells then you cannot use it anywhere but in that notebook...
You don't seem to understand how notebooks can be used and processed themselves. They are just data and there are libraries for loading and transforming them. You might check out pydev and papermill to get an idea about this (and maybe also literate programming in general).
I do all of my code in notebooks. If your docstring is slow to read, it is a bad docstring. To be clear, a docstring is the thing you get when you have a function, you don't know what it does, and you press shift tab to bring up the brief description of what it does in a pop up (in jupyter lab). It should be the quickest, least error prone form of documentation because it is available on demand and is supposed to be short.
Using notebooks as functions is a bad idea. I think papermill is trash.
Notebooks are good for illustrating what you did and what you got. They are not good for illustrating how you did it.
It truly is fascinating that you continue to insist that I don't know what a docstring is. Notebooks are good for certain things and not good for others.
This conversation is going nowhere.
