Skip to content

Commit

Permalink
Merge pull request #68 from timvink/v2_version
Browse files Browse the repository at this point in the history
Refactor for compatibility with macros
  • Loading branch information
timvink authored Aug 15, 2024
2 parents 74817da + 80928d8 commit 9f6b419
Show file tree
Hide file tree
Showing 25 changed files with 302 additions and 198 deletions.
4 changes: 2 additions & 2 deletions .github/workflows/pythonpublish.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,10 +13,10 @@ jobs:
id-token: write
contents: write
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4

- name: Set up Python
uses: actions/setup-python@v4
uses: actions/setup-python@v5
with:
python-version: '3.x'

Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/workflow.yml
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ jobs:
- name: Upload coverage to Codecov
if: contains(env.USING_COVERAGE, matrix.python-version) && github.ref == 'refs/heads/master'
uses: codecov/codecov-action@v1
uses: codecov/codecov-action@v4
with:
token: ${{ secrets.CODECOV_TOKEN }}
file: ./coverage.xml
Expand Down
2 changes: 1 addition & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,4 +49,4 @@ git tag <version>
git push origin <version>
```

Then manually create a github release to trigger publishing to pypi.
Then manually create a github release to trigger publishing to pypi.
5 changes: 3 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,9 +38,10 @@ In your markdown files you can now use:
{{ read_csv('path_to_table.csv') }}
```

Where the path is relative to the location of your project's `mkdocs.yml` file (although you can [change that](https://timvink.github.io/mkdocs-table-reader-plugin/options) to be relative to your `docs/` directory).
Where the path is relative to the location of your project's `mkdocs.yml` file, _or_ your project's `docs/` directory, _or_ the location of your markdown source file (all 3 possible locations will be searched, in that order).

- There are [readers](https://timvink.github.io/mkdocs-table-reader-plugin/readers/) for `.csv`, `.fwf`, `.json`, `.xlsx`, `.yaml` and `.tsv` files. There is also the `read_raw()` reader that will allow you to insert tables (or other content) already in markdown format.
- There are [readers](https://timvink.github.io/mkdocs-table-reader-plugin/readers/) for `.csv`, `.fwf`, `.json`, `.xls`, `.xlsx`, `.yaml`, `.feather` and `.tsv` files. There is also the `read_raw()` reader that will allow you to insert tables (or other content) already in markdown format.
- `table-reader` is compatible with [`mkdocs-macros-plugin`](https://mkdocs-macros-plugin.readthedocs.io/en/latest/), which means you can [dynamically insert tables using jinja2 syntax](https://timvink.github.io/mkdocs-table-reader-plugin/howto/use_jinja2/).

## Documentation and how-to guides

Expand Down
48 changes: 10 additions & 38 deletions docs/howto/project_structure.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,11 +26,15 @@ If you only want to include an occasional table in a specific markdown file, jus
\{\{ read_csv("another_table.csv") \}\}
```

This works because the [option](../options.md) `search_page_directory` defaults to `True`.
In `page.md`, to read `another_table.csv`, you can choose to use:

- <code>\{\{ read_csv("docs/folder/another_table.csv") \}\}</code> (Path relative to mkdocs.yml)
- <code>\{\{ read_csv("folder/another_table.csv") \}\}</code> (Path relative to docs/ directory)
- <code>\{\{ read_csv("another_table.csv") \}\}</code> (Path relative to page source file)

## Re-using tables across markdown files

If you want to reuse tables in multiple markdown files, you'll want to store them in a central directory, like `docs/assets/tables`.
If you want to reuse tables in multiple markdown files, or have many tables, you'll want to store them in a central directory, like `docs/assets/tables`.
That way, if you restructure your navigation, the links to the tables won't break either.
It's also great if you generate tables because the output directory will be the same.

Expand All @@ -45,44 +49,12 @@ Given the following project structure:
│ └── assets/
│ └── tables/
│ └── another_table.csv
│ └── page.md
└── mkdocs.yml
```

In `page.md`, to read `basic_table.csv`, you can choose to use:

- <code>\{\{ read_csv("docs/assets/tables/another_table.csv") \}\}</code> when `base_path` [option](../options.md) is set to `config_dir` (default)
- <code>\{\{ read_csv("assets/tables/another_table.csv") \}\}</code> when `base_path` [option](../options.md) is set to `docs_dir`
- <code>\{\{ read_csv("../another_table.csv") \}\}</code> if you want to use a relative path and `search_page_directory` [option](../options.md) is enabled (default). Note that `..` stands for "one directory up".

## A central table directory combined with same-directory tables

If you have some central tables that you want to reuse, and some tables that are specific to a page, you could use the following project structure:

```nohighlight
.
├── docs/
│ ├── tables/
│ | └── basic_table.csv
│ └── folder/
│ └── another_table.csv
│ └── page.md
└── mkdocs.yml
```

In `page.md`, to read `basic_table.csv`, you can choose to use:

- <code>\{\{ read_csv("docs/tables/basic_table.csv") \}\}</code> when `base_path` [option](../options.md) is set to `config_dir` (default)
- <code>\{\{ read_csv("tables/basic_table.csv") \}\}</code> when `base_path` [option](../options.md) is set to `docs_dir`
- <code>\{\{ read_csv("basic_table.csv") \}\}</code> when:
- `bash_path` [option](../options.md) is set to `config_dir` and `data_path` is set to `docs/tables`, OR
- `bash_path` [option](../options.md) is set to `docs_dir` and `data_path` is set to `tables`

In `page.md`, to read `another_table.csv`, you can choose to use:

- <code>\{\{ read_csv("docs/folder/another_table.csv") \}\}</code> when `base_path` is set to `config_dir` (default)
- <code>\{\{ read_csv("folder/another_table.csv") \}\}</code> when `base_path` is set to `docs_dir`
- <code>\{\{ read_csv("another_table.csv") \}\}</code> when:
- `search_page_directory` [option](../options.md) is enabled (default), OR
- `bash_path` [option](../options.md) is set to `config_dir` and `data_path` is set to `docs/folder`, OR
- `bash_path` [option](../options.md) is set to `docs_dir` and `data_path` is set to `folder`
- <code>\{\{ read_csv("docs/assets/tables/another_table.csv") \}\}</code> (Path relative to mkdocs.yml)
- <code>\{\{ read_csv("assets/tables/another_table.csv") \}\}</code> (Path relative to docs/ directory)
- <code>\{\{ read_csv("../assets/tables/another_table.csv") \}\}</code> (Path relative to page source file _(note that `..` stands for "one directory up")_)

26 changes: 26 additions & 0 deletions docs/howto/use_jinja2.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# Using jinja2

`table-reader` supports [`mkdocs-macros-plugin`](https://mkdocs-macros-plugin.readthedocs.io/en/latest/), which enables you to use jinja2 syntax inside markdown files (among other things).

To enable `macros`, specify the plugin _before_ `table-reader` in your `mkdocs.yml` file:

```yaml
plugins:
- macros
- table-reader
```
Now you can do cool things like dynamically load a list of tables:
```markdown
# index.md

{% set table_names = ["basic_table.csv","basic_table2.csv"] %}
{% for table_name in table_names %}

{ { read_csv(table_name) }}

{% endfor %}

```
36 changes: 4 additions & 32 deletions docs/options.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,63 +10,35 @@ You can customize the plugin by setting options in `mkdocs.yml`. For example:
```yml
plugins:
- table-reader:
base_path: "config_dir"
data_path: "."
search_page_directory: True
allow_missing_files: False
select_readers:
- read_csv
- read_json
enabled: True
```
## `base_path`

The base path where `mkdocs-table-reader-plugin` will search for input files. The path to your table files should be relative to the `base_path`. Allowed values:

- `config_dir` (default): the directory where your project's `mkdocs.yml` file is located.
- `docs_dir`: the directory where your projects' `docs/` folder is located.

If you store your table in `docs/assets/table.csv`, you can insert it in any markdown page using <code>\{\{ read_csv("docs/assets/table.csv") \}\}</code>, or when `base_path` is `docs_dir`, with <code>\{\{ read_csv("assets/table.csv") \}\}</code>.

!!! info

Note that by default the plugin will _also_ search the page's directory but only when a table is not found.

For more examples see the how to guide on [project structure](howto/project_structure.md).

## `data_path`

The path to your table files should be relative to the `base_path`. If you use a folder for all your table files you can shorten the path specification by setting the `data_path`.
Default is `.`. Set a default path to the searched directories in order to shorten table filename specifications.

For example, if your table is located in `docs/tables/basic_table.csv`, you can set `data_path` to `docs/tables/` and leave `base_path` to the default `config_dir`. Then you will be able to use <code>\{\{ read_csv("basic_table.csv") \}\}</code> instead of <code>\{\{ read_csv("docs/tables/basic_table.csv") \}\}</code> inside any markdown page.
Given a file path, `table-reader` will search for that file relative to your your project's `mkdocs.yml` and relative to your `docs/` folder. If you use a folder for all your table files you can shorten the path specification by setting the `data_path`.

Default is `.`, which means you need to specify the path to your table files relative to the `base_path`.
For example, if your table is located in `docs/assets/tables/basic_table.csv`, you can set `data_path` to `docs/assets/tables/`. Then you will be able to use <code>\{\{ read_csv("basic_table.csv") \}\}</code> instead of <code>\{\{ read_csv("docs/assets/tables/basic_table.csv") \}\}</code> inside any markdown page.

!!! info

Note that by default the plugin will _also_ search the page's directory but only when a table is not found.

For more examples see the how to guide on [project structure](howto/project_structure.md).

## `search_page_directory`

Default: `True`. When enabled, if a filepath is not found, the plugin will also attempt to find the file relative to the current page's directory.

!!! info

Note that even when `True`, the path relative to `data_path` is searched first,
and if a file is not found there, then the page's directory is searched.

For more examples see the how to guide on [project structure](howto/project_structure.md).

## `allow_missing_files`

Default: `False`. When enabled, if a filepath is not found, the plugin will raise a warning instead of an error.

## `select_readers`

Default: Selects all available readers. Specify a list of reader to improve documentation build times for large sites.
Default: Selects all available readers. Specify a list of readers to improve documentation build times for very large sites.

## `enabled`

Expand Down
15 changes: 1 addition & 14 deletions docs/schema.json
Original file line number Diff line number Diff line change
Expand Up @@ -13,25 +13,12 @@
"markdownDescription": "https://timvink.github.io/mkdocs-table-reader-plugin/options/",
"type": "object",
"properties": {
"base_path": {
"title": "The base path where mkdocs-table-reader-plugin will search for input files. The path to your table files should be relative to the base_path.",
"markdownDescription": "https://timvink.github.io/mkdocs-table-reader-plugin/options/#base_path",
"type": "string",
"enum": ["config_dir", "docs_dir"],
"default": "config_dir"
},
"data_path": {
"title": "The path to your table files should be relative to the base_path. If you use a folder for all your table files you can shorten the path specification by setting the data_path.",
"title": "Additional path to search",
"markdownDescription": "https://timvink.github.io/mkdocs-table-reader-plugin/options/#data_path",
"type": "string",
"default": "."
},
"search_page_directory": {
"title": "When enabled, if a filepath is not found, the plugin will also attempt to find the file relative to the current page's directory.",
"markdownDescription": "https://timvink.github.io/mkdocs-table-reader-plugin/options/#search_page_directory",
"type": "boolean",
"default": true
},
"allow_missing_files": {
"title": "When enabled, if a filepath is not found, the plugin will raise a warning instead of an error.",
"markdownDescription": "https://timvink.github.io/mkdocs-table-reader-plugin/options/#allow_missing_files",
Expand Down
6 changes: 4 additions & 2 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ nav:
- How to:
- howto/customize_tables.md
- howto/preprocess_tables.md
- howto/use_jinja2.md
- howto/project_structure.md
- howto/docker.md
- howto/alternatives.md
Expand Down Expand Up @@ -50,8 +51,8 @@ plugins:
- table-reader:
data_path: "docs/assets"
- git-authors:
exclude:
- index.md
exclude:
- index.md
- git-revision-date-localized:
type: timeago
timezone: Europe/Amsterdam
Expand All @@ -64,6 +65,7 @@ markdown_extensions:
- meta
- admonition
- pymdownx.keys
- pymdownx.escapeall
- pymdownx.highlight
- pymdownx.inlinehilite
- pymdownx.snippets
Expand Down
15 changes: 10 additions & 5 deletions mkdocs_table_reader_plugin/markdown.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,11 @@
import pandas as pd
import textwrap


def replace_unescaped_pipes(text: str) -> str:
"""
Replace unescaped pipes.
For regex explanation, see https://regex101.com/r/s8H588/1
Args:
Expand All @@ -24,20 +25,24 @@ def convert_to_md_table(df: pd.DataFrame, markdown_kwargs: Dict) -> str:
"""
# Escape any pipe characters, | to \|
# See https://github.com/astanin/python-tabulate/issues/241
df.columns = [replace_unescaped_pipes(c) if isinstance(c, str) else c for c in df.columns]
df.columns = [
replace_unescaped_pipes(c) if isinstance(c, str) else c for c in df.columns
]

# Avoid deprecated applymap warning on pandas>=2.0
# See https://github.com/timvink/mkdocs-table-reader-plugin/issues/55
if pd.__version__ >= "2.1.0":
df = df.map(lambda s: replace_unescaped_pipes(s) if isinstance(s, str) else s)
else:
df = df.applymap(lambda s: replace_unescaped_pipes(s) if isinstance(s, str) else s)
df = df.applymap(
lambda s: replace_unescaped_pipes(s) if isinstance(s, str) else s
)

if "index" not in markdown_kwargs:
markdown_kwargs["index"] = False
if "tablefmt" not in markdown_kwargs:
markdown_kwargs["tablefmt"] = "pipe"

return df.to_markdown(**markdown_kwargs)


Expand All @@ -56,7 +61,7 @@ def fix_indentation(leading_spaces: str, text: str) -> str:
leading_spaces = int(len(leading_spaces) / 4) * " "

fixed_lines = []
for line in text.split('\n'):
for line in text.split("\n"):
fixed_lines.append(textwrap.indent(line, leading_spaces))
text = "\n".join(fixed_lines)
return text
Loading

0 comments on commit 9f6b419

Please sign in to comment.