Examples

This document goes over some usages of remarking.

Note: we use jq tool here to format the output json. This is not needed if you’re piping to another command or to a file.

JSON output

Run remarking over a folder called books in the reMarkable cloud.

> remarking run json books | jq
Extractors: remarkable
Collections: books
Connecting to RM cloud
Connected to RM cloud.
Retrieving cloud metadata
Downloading documents
Downloading "Through the Looking Glass"
Downloaded 1 documents.
Running extractors on documents
Running extractor "RemarkableHighlightExtractor" on "Through the Looking Glass"
Ran extractors and found 1 highlights, 1 are new.
{
  "documents": [
    {
      "id": "2b329909-332d-4c28-b6c1-d298227cc82c",
      "version": 7,
      "modified_client": 1626736737,
      "type": "DocumentType",
      "name": "Through the Looking Glass",
      "current_page": 12,
      "bookmarked": false,
      "parent": "d29d67f9-faff-429c-b800-b7815173dcb2"
    }
  ],
  "highlights": [
    {
      "hash": "14235eca5db4a758ad34ab483df737714d10e710a3e3277d726a64da",
      "document_id": "2b329909-332d-4c28-b6c1-d298227cc82c",
      "text": "Alice was sitting curled up in a corner of the great arm-chair",
      "page_number": 11,
      "extracted_at": 1626993849,
      "extraction_method": "RemarkableHighlightExtractor"
    }
  ]
}

JSON output to file

Run remarking over the books folder and write the resulting highlights to file named highlights.json

> remarking run json -o highlights.json books | jq
Extractors: remarkable
Collections: books
Connecting to RM cloud
Connected to RM cloud.
Retrieving cloud metadata
Downloading documents
Downloading "Through the Looking Glass"
Downloaded 1 documents.
Running extractors on documents
Running extractor "RemarkableHighlightExtractor" on "Through the Looking Glass"
Ran extractors and found 1 highlights, 1 are new.

The contents of highlights.json

{"documents": [{"id": "2b329909-332d-4c28-b6c1-d298227cc82c", "version": 7, "modified_client": 1626736737, "type": "DocumentType", "name": "Through the Looking Glass", "current_page": 12, "bookmarked": false, "parent": "d29d67f9-faff-429c-b800-b7815173dcb2"}], "highlights": [{"hash": "14235eca5db4a758ad34ab483df737714d10e710a3e3277d726a64da", "document_id": "2b329909-332d-4c28-b6c1-d298227cc82c", "text": "Alice was sitting curled up in a corner of the great arm-chair", "page_number": 11, "extracted_at": 1626993854, "extraction_method": "RemarkableHighlightExtractor"}]}

CSV output

Run remarking over the books folder and output the discovered highlights as a csv.

> remarking run csv books
Extractors: remarkable
Collections: books
Connecting to RM cloud
Connected to RM cloud.
Retrieving cloud metadata
Downloading documents
Downloading "Through the Looking Glass"
Downloaded 1 documents.
Running extractors on documents
Running extractor "RemarkableHighlightExtractor" on "Through the Looking Glass"
Ran extractors and found 1 highlights, 1 are new.
highlight_text,highlight_page_number,document_name
Alice was sitting curled up in a corner of the great arm-chair,11,Through the Looking Glass

CSV output to file

Run remarking over the books folder and output the resulting highlights as a csv to a file called highlights.csv file.

> remarking run csv -o highlights.csv books
Extractors: remarkable
Collections: books
Connecting to RM cloud
Connected to RM cloud.
Retrieving cloud metadata
Downloading documents
Downloading "Through the Looking Glass"
Downloaded 1 documents.
Running extractors on documents
Running extractor "RemarkableHighlightExtractor" on "Through the Looking Glass"
Ran extractors and found 1 highlights, 1 are new.

The contents of highlights.csv

highlight_text,highlight_page_number,document_name
Alice was sitting curled up in a corner of the great arm-chair,11,Through the Looking Glass

Custom CSV output to file

Run remarking over the books folder and output the resulting highlights as a csv to a file called highlights.csv file. The csv will be delimited with a |.

> remarking run csv -o highlights.csv --delimiter '|' books
Extractors: remarkable
Collections: books
Connecting to RM cloud
Connected to RM cloud.
Retrieving cloud metadata
Downloading documents
Downloading "Through the Looking Glass"
Downloaded 1 documents.
Running extractors on documents
Running extractor "RemarkableHighlightExtractor" on "Through the Looking Glass"
Ran extractors and found 1 highlights, 1 are new.

The contents of highlights.csv

highlight_text|highlight_page_number|document_name
Alice was sitting curled up in a corner of the great arm-chair|11|Through the Looking Glass

Table output

Print out a table that contains the highlights for all documents in the books folder.

> remarking run table books
Extractors: remarkable
Collections: books
Connecting to RM cloud
Connected to RM cloud.
Retrieving cloud metadata
Downloading documents
Downloading "Through the Looking Glass"
Downloaded 1 documents.
Running extractors on documents
Running extractor "RemarkableHighlightExtractor" on "Through the Looking Glass"
Ran extractors and found 1 highlights, 1 are new.
highlight_text                                                    highlight_page_number  document_name
--------------------------------------------------------------  -----------------------  -------------------------
Alice was sitting curled up in a corner of the great arm-chair                       11  Through the Looking Glass

Plain table output

Print out a table that contains the highlights for all documents in the books folder. When printing plain only whitespace separates columns and newlines for each row.

> remarking run table --plain books
Extractors: remarkable
Collections: books
Connecting to RM cloud
Connected to RM cloud.
Retrieving cloud metadata
Downloading documents
Downloading "Through the Looking Glass"
Downloaded 1 documents.
Running extractors on documents
Running extractor "RemarkableHighlightExtractor" on "Through the Looking Glass"
Ran extractors and found 1 highlights, 1 are new.
highlight_text                                                    highlight_page_number  document_name
Alice was sitting curled up in a corner of the great arm-chair                       11  Through the Looking Glass

Specify extractors

Use the remarkable extractor to extract highlights from documents. You can see all available extractors by running remarking list extractors. remarkable is the default value for --extractors option.

> remarking run json --extractors remarkable books | jq
Extractors: remarkable
Collections: books
Connecting to RM cloud
Connected to RM cloud.
Retrieving cloud metadata
Downloading documents
Downloading "Through the Looking Glass"
Downloaded 1 documents.
Running extractors on documents
Running extractor "RemarkableHighlightExtractor" on "Through the Looking Glass"
Ran extractors and found 1 highlights, 1 are new.
{
  "documents": [
    {
      "id": "2b329909-332d-4c28-b6c1-d298227cc82c",
      "version": 7,
      "modified_client": 1626736737,
      "type": "DocumentType",
      "name": "Through the Looking Glass",
      "current_page": 12,
      "bookmarked": false,
      "parent": "d29d67f9-faff-429c-b800-b7815173dcb2"
    }
  ],
  "highlights": [
    {
      "hash": "14235eca5db4a758ad34ab483df737714d10e710a3e3277d726a64da",
      "document_id": "2b329909-332d-4c28-b6c1-d298227cc82c",
      "text": "Alice was sitting curled up in a corner of the great arm-chair",
      "page_number": 11,
      "extracted_at": 1626993883,
      "extraction_method": "RemarkableHighlightExtractor"
    }
  ]
}

Persist

Use a database to keep track of documents seen, and highlights produced. When using the persist command, the last modified date of documents in the Remarkable cloud will be used to trigger their processing. highlights are deduped by checking their text and document. By default, if a --sqlalchemy argument is passed, a sqlite file is created in the current working directory called remarking.sqlite3. The persist argument will only output highlights that are considered new according to the database state.

> remarking persist json books | jq
Extractors: remarkable
Collections: books
Connecting to RM cloud
Connected to RM cloud.
Retrieving cloud metadata
Downloading documents
Downloaded 0 documents.
Running extractors on documents
Ran extractors and found 0 highlights, 0 are new.
{
  "documents": [],
  "highlights": []
}

Persist to SQLite

When passing the --sqlalchemy option, persist will use the option to create a sqlalchemy engine. This is particularly useful for syncing with an external database. Check the sqlalchemy documentation for more info on sqlalchemy connection string. You can also set the REMARKING_PERSIST_SQALCHEMY env var instead of the --sqlalchemy option.

> remarking persist --sqlalchemy sqlite:///my_database.sqlite3 json books | jq
Extractors: remarkable
Collections: books
Connecting to RM cloud
Connected to RM cloud.
Retrieving cloud metadata
Downloading documents
Downloading "Through the Looking Glass"
Downloaded 1 documents.
Running extractors on documents
Running extractor "RemarkableHighlightExtractor" on "Through the Looking Glass"
Ran extractors and found 1 highlights, 1 are new.
{
  "documents": [
    {
      "id": "2b329909-332d-4c28-b6c1-d298227cc82c",
      "version": 7,
      "modified_client": 1626736737,
      "type": "DocumentType",
      "name": "Through the Looking Glass",
      "current_page": 12,
      "bookmarked": false,
      "parent": "d29d67f9-faff-429c-b800-b7815173dcb2"
    }
  ],
  "highlights": [
    {
      "hash": "14235eca5db4a758ad34ab483df737714d10e710a3e3277d726a64da",
      "document_id": "2b329909-332d-4c28-b6c1-d298227cc82c",
      "text": "Alice was sitting curled up in a corner of the great arm-chair",
      "page_number": 11,
      "extracted_at": 1626993891,
      "extraction_method": "RemarkableHighlightExtractor"
    }
  ]
}

Persist to MySQL

Use the mysql database located at host for state management. You can query this database directly to extract all historical highlights and documents.

> remarking persist --sqlalchemy mysql+pymysql://user:pass@host/dbname?charset=utf8mb4 json books | jq
    Extractors: remarkable
    Collections: books
    Connecting to RM cloud
    Connected to RM cloud.
    Retrieving cloud metadata
    Downloading documents
    Downloading "Through the Looking Glass"
    Downloaded 1 documents.
    Running extractors on documents
    Running extractor "RemarkableHighlightExtractor" on "Through the Looking Glass"
    Ran extractors and found 1 highlights, 1 are new.
    {
      "documents": [
        {
          "id": "2b329909-332d-4c28-b6c1-d298227cc82c",
          "version": 7,
          "modified_client": 1626736737,
          "type": "DocumentType",
          "name": "Through the Looking Glass",
          "current_page": 12,
          "bookmarked": false,
          "parent": "d29d67f9-faff-429c-b800-b7815173dcb2"
        }
      ],
      "highlights": [
        {
          "hash": "14235eca5db4a758ad34ab483df737714d10e710a3e3277d726a64da",
          "document_id": "2b329909-332d-4c28-b6c1-d298227cc82c",
          "text": "Alice was sitting curled up in a corner of the great arm-chair",
          "page_number": 11,
          "extracted_at": 1626993891,
          "extraction_method": "RemarkableHighlightExtractor"
        }
      ]
    }

Quiet logging

Quiet any logging with -q Only the results are output stdout.

> remarking run json -q books | jq
{
  "documents": [
    {
      "id": "2b329909-332d-4c28-b6c1-d298227cc82c",
      "version": 7,
      "modified_client": 1626736737,
      "type": "DocumentType",
      "name": "Through the Looking Glass",
      "current_page": 12,
      "bookmarked": false,
      "parent": "d29d67f9-faff-429c-b800-b7815173dcb2"
    }
  ],
  "highlights": [
    {
      "hash": "14235eca5db4a758ad34ab483df737714d10e710a3e3277d726a64da",
      "document_id": "2b329909-332d-4c28-b6c1-d298227cc82c",
      "text": "Alice was sitting curled up in a corner of the great arm-chair",
      "page_number": 11,
      "extracted_at": 1626993896,
      "extraction_method": "RemarkableHighlightExtractor"
    }
  ]
}