Installation and Getting Started¶

Python: 3.7+
Platforms: Linux, Windows, MacOS
PyPI package name: remarking

Remarking makes it easy to extract highlights from the ReMarkable cloud.

Install `remarking`¶

Run the following in your command line:

pip install -U remarking

Check the version you have installed

remarking --version

Download your first highlights¶

Before you download your first highlights you will need to make sure you have actually annotated a document on your reMarkable.

You can annotate a document by highlighting text on the ReMarkable using the highlight pen.

Once you have some highlighted text you can start extracting it.

Note

Remarking requires a one-time authorization token from https://my.remarkable.com/device/desktop/connect in order to access the documents on the reMarkable cloud.

You will be prompted to enter the token if remarking is not able to authorize with the cloud.

Remarking uses rmapy to authenticate and access documents on the cloud. rmapy stores a token in ~/.rmapi so that you do not need to re-authenticate on each invocation of Remarking.

With a document highlighted, you can run remarking either on the document itself or the folder that contains the document.

Let’s say the document is called Through the Looking Glass and it lives in the library folder:

> remarking run table library | jq
Extractors: remarkable
Collections: library
Connecting to RM cloud
Connected to RM cloud.
Retrieving cloud metadata
Downloading documents
Downloading "Through the Looking Glass"
Downloaded 1 documents.
Running extractors on documents
Running extractor "RemarkableHighlightExtractor" on "Through the Looking Glass"
Ran extractors and found 1 highlights, 1 are new.
{
  "documents": [
    {
      "id": "2b329909-332d-4c28-b6c1-d298227cc82c",
      "version": 5,
      "modified_client": 1626736734,
      "type": "DocumentType",
      "name": "Through the Looking Glass",
      "current_page": 11,
      "bookmarked": false,
      "parent": "d29d67f9-faff-429c-b800-b7815173dcb2"
    }
  ],
  "highlights": [
    {
      "hash": "14235eca5db4a758ad34ab483df737714d10e710a3e3277d726a64da",
      "document_id": "2b329909-332d-4c28-b6c1-d298227cc82c",
      "text": "Alice was sitting curled up in a corner of the great arm-chair",
      "page_number": 11,
      "extracted_at": 1626723209,
      "extraction_method": "RemarkableHighlightExtractor"
    }
  ]
}

You could also have done:

> remarking run table "Through the Looking Glass"
Extractors: remarkable
Collections: library
Connecting to RM cloud
Connected to RM cloud.
Retrieving cloud metadata
Downloading documents
Downloading "Through the Looking Glass"
Downloaded 1 documents.
Running extractors on documents
Running extractor "RemarkableHighlightExtractor" on "Through the Looking Glass"
highlight_text                                                    highlight_page_number  document_name
--------------------------------------------------------------  -----------------------  -------------------------
Alice was sitting curled up in a corner of the great arm-chair                       11  Through the Looking Glass

The Structure of a Remarking Command¶

All remarking commands follow a similar structure.

remarking <mode command> [mode options] <writer command> [options] args

Modes¶

The first command is called the mode command. It is one of:

run
persist

Writer Commands¶

The second command specifies which the writer command to use. The built-in writer commands are:

json
csv
table

Learn more about them below in Writer Commands.

The documentation section WriterCommand Guide explains how to add custom writer commands.

Run¶

The run mode will download all highlights for a given document or folder and output them all according to the writer you have set.

> remarking run table "Through the Looking Glass"
Extractors: remarkable
Collections: library
Connecting to RM cloud
Connected to RM cloud.
Retrieving cloud metadata
Downloading documents
Downloading "Through the Looking Glass"
Downloaded 1 documents.
Running extractors on documents
Running extractor "RemarkableHighlightExtractor" on "Through the Looking Glass"
highlight_text                                                    highlight_page_number  document_name
--------------------------------------------------------------  -----------------------  -------------------------
Alice was sitting curled up in a corner of the great arm-chair                       11  Through the Looking Glass

The run mode will always return the same highlights, it does not manage any state.

The persist run mode, on the other hand, does manage state.

Persist¶

Remarking shines when you want to keep track of highlights over time.

The persist mode makes that easy to do.

persist uses the sqlalchemy ORM to map highlights and documents to any SQLAlchemy supported backing datastore.

SqlAlchemy defaults¶

By default, remarking will persist to a local sqlite3 database called remarking_database.sqlite3 that is stored in the current working directory.

Providing an argument for –sqlalchemy allows you to use any database that sqlalchemy supports as a backing database.

SqlAlchemy dialects¶

Some examples of the many listed dialects on the sqlalchemy website:

MySQL
PostgreSQL
SQLite

Warning

You need to install the specific driver for database you would like to connect to.

For example, to connect to a MySQL database, you need to make sure a mysql driver is accessible by sqlalchemy.

On most systems you will need the mysqlclient binaries installed on your machine which can then allow you to install the required python package:

pip install mysqlclient

Please refer to the sqlalchemy docs for more info

An example of connecting to a MySQL database:

> remarking persist --sqlalchemy mysql+pymysql://user:pass@host/dbname?charset=utf8mb4 table library
Extractors: remarkable
Collections: library
Connecting to RM cloud
Connected to RM cloud.
Retrieving cloud metadata
Downloading documents
Downloading "Through the Looking Glass"
Downloaded 1 documents.
Running extractors on documents
Running extractor "RemarkableHighlightExtractor" on "Through the Looking Glass"
Ran extractors and found 1 highlights, 1 are new.
highlight_text                                                    highlight_page_number  document_name
--------------------------------------------------------------  -----------------------  -------------------------
Alice was sitting curled up in a corner of the great arm-chair                       11  Through the Looking Glass

Querying the database will show a documents and highlights table that can be queried and futher used.

Another execution of persist without modifying the document would yield no new highlights.

If the document is modified and a new highlight is made, only the new highlight will be returned on the next run of persist

Setting the `--sqlalchemy` option¶

If the argument provided for the --sqlalchemy is a file, then the first line of that file is read and used as the sqlalchemy connection string. This is useful to avoid leaking secrets.

You can also set the REMARKING_SQLALCHEMY env var instead of the cmd line option.

Extractors¶

Extractors are what actually parse a document and return highlights for it.

You can specify the extractors you want to use via the --extractors option for both run and persist.

Listing extractors¶

You can list all extractors available to use with

remarking list extractors

Adding new extractors¶

Check out Write an Extractor for how to create your own extractor.

Writer Commands¶

Writer commands as mentioned in the The Structure of a Remarking Command section are the second part of a remarking command.

Writer commands specify how the highlights should be output.

Viewing Writer Commands¶

You can see all available writer commands with:

remarking run --help

Built-in Writer Commands¶

Remember that run will always output all highlights found and that persist will only output new highlights found since the last execution.

json¶

Usage: remarking run json [OPTIONS] [COLLECTION_NAMES]...

  Output highlights and documents as JSON

Options:
  -t, --token TEXT                One time auth token from for the reMarkable
                                  cloud. Needs only be specified once.  [env
                                  var: REMARKING_TOKEN]
  -e, --extractors TEXT           Comma delimited list of extractors to use.
                                  Run `remarking list extractors` to see valid
                                  extractors.  [default: remarkable]
  -o, --output FILENAME           Output highlights to the given file
  -w, --working-directory DIRECTORY
                                  Working directory where files will be
                                  downloaded and highlights generated.
                                  [default: (A randomly generated path within
                                  /tmp/)]
  -q, --quiet                     Print nothing.
  -h, --help                      Show this message and exit.

csv¶

Usage: remarking run csv [OPTIONS] [COLLECTION_NAMES]...

  Output highlights normalized with documents as csv.

  Check out `remarking list columns` for a list of columns to choose from for
  the `--columns` option.



Options:
  -t, --token TEXT                One time auth token from for the reMarkable
                                  cloud. Needs only be specified once.  [env
                                  var: REMARKING_TOKEN]
  -e, --extractors TEXT           Comma delimited list of extractors to use.
                                  Run `remarking list extractors` to see valid
                                  extractors.  [default: remarkable]
  -o, --output FILENAME           Output highlights to the given file
  -w, --working-directory DIRECTORY
                                  Working directory where files will be
                                  downloaded and highlights generated.
                                  [default: (A randomly generated path within
                                  /tmp/)]
  -q, --quiet                     Print nothing.
  --delimiter TEXT                Delimiter to use to split columns
  --columns TEXT                  Comma delimited list of columns to print
                                  when using plain printing. `remarking list
                                  columns` shows all available columns
                                  [default: highlight_text,document_name,highl
                                  ight_page_number]
  -h, --help                      Show this message and exit.

table¶

Usage: remarking run table [OPTIONS] [COLLECTION_NAMES]...

  Output highlights normalized with documents as a table.

  Check out `remarking list columns` for a list of columns to choose from for
  the `--columns` option.

Options:
  -t, --token TEXT                One time auth token from for the reMarkable
                                  cloud. Needs only be specified once.  [env
                                  var: REMARKING_TOKEN]
  -e, --extractors TEXT           Comma delimited list of extractors to use.
                                  Run `remarking list extractors` to see valid
                                  extractors.  [default: remarkable]
  -o, --output FILENAME           Output highlights to the given file
  -w, --working-directory DIRECTORY
                                  Working directory where files will be
                                  downloaded and highlights generated.
                                  [default: (A randomly generated path within
                                  /tmp/)]
  -q, --quiet                     Print nothing.
  --truncate / --no-truncate      Truncate results when printing plain
  --plain / --no-plain            Output one data entry per line.
  --columns TEXT                  Comma delimited list of columns to print
                                  when using plain printing. `remarking list
                                  columns` shows all available columns
                                  [default: highlight_text,document_name,highl
                                  ight_page_number]
  -h, --help                      Show this message and exit.

Installation and Getting Started¶

Install remarking¶

Download your first highlights¶

The Structure of a Remarking Command¶

Modes¶

Writer Commands¶

Run¶

Persist¶

SqlAlchemy defaults¶

SqlAlchemy dialects¶

Setting the --sqlalchemy option¶

Extractors¶

Listing extractors¶

Adding new extractors¶

Writer Commands¶

Viewing Writer Commands¶

Built-in Writer Commands¶

json¶

csv¶

table¶

Install `remarking`¶

Setting the `--sqlalchemy` option¶