Home

Apache Daffodil VS Code Extension

This is a Visual Studio Code (VS Code) extension which enables the interactive debugging of Data Format Description Language (DFDL) schema parsing using Apache Daffodil. DFDL, also known as Daffodil, is a data modeling language used to describe file formats. The DFDL modeling language is a subset of XML Schema Definition (XSD). Just as file formats are rich and complex, so is the modeling language to describe them. Developing DFDL schemas can be a huge challenge, requiring a lot of iterative development, and testing. The purpose of these tools are to ease the burden on DFDL schema developers, enabling them to develop high quality, high resolution DFDL schemas, in less time. Developing the tools for Visual Studio Code (VS Code) seems the natural choice as VS Code is free, open source, cross-platform, well-maintained, extensible, and ubiquitous in the developer community.

Current tools bundled in the extension

• Daffodil Syntax Highlighting

The DFDL modeling language is rich and complex. Developers using modern IDEs expect some degree of built-in language support for the language they are developing in, and DFDL should be no different. The tools provide syntax highlighting to improve the readability and context of the text in addition to feedback to the developer indicating at least the structure and code appear syntactically correct.

• Daffodil Schema Code Completion

Code completion, also known as “Intellisense”, offers context-aware code segment predictions that can speed up schema development by reducing keyboard input, reduce memorization by the developer, and reduce typos.

• Daffodil Data Parse Debugger

The debugger allows the developer to carefully control the execution of a Daffodil file parse. Given a DFDL schema and a target data file, the developer can step through the execution of a parse line by line, or until the parse reaches some developer-defined line, known as a break point, in the schema. What is particularly helpful is that the developer can watch the parsed output, known as the infoset, as it’s being created by the parser. This enables the developer to quickly discover and correct issues, improving the development and testing cycles.

• Daffodil Data Editor

The integrated data editor is a new experimental feature that is under heavy development. It is akin to a hex editor, but tuned specifically for challenging Daffodil use cases. It is designed from scratch to handle virtually any sized file, well beyond the limits of Visual Studio Code’s standard text editor, and of course it can handle non-text data just as well as text data. It will have support for setting Daffodil debugger breakpoints on offsets in the data file in addition to line positions in the DFDL schema. It will handle non-standard byte sizes and non-aligned bytes. The least significant byte can be the first or last bit in a byte. As an editor designed for Daffodil developers by Daffodil developers, features of the tool can evolve quickly to address the specific needs of the Daffodil community.

Prerequisites

This guide assumes you have Visual Studio Code (VS Code) and a Java Runtime Environment installed.

• Install VS Code
• Install Java Runtime 11 or greater

Installing the Daffodil extension

Until the extension is available in the VS Code Extension Marketplace, please download the latest .vsix file from the releases page. Then install it by either:

• using the "Extensions: Install from VSIX" command from within VS Code by opening the Command Palette (Mac = Command+Shift+P, Windows/Linux = Ctrl+Shift+P), and typing vsix to bring up the command and pointing it at the downloaded vsix file; or
• on the command-line via code --install-extension <path-to-downloaded-vsix>.

Debugging a DFDL schema

Debug configuration

Debugging a schema needs both the schema to use and a data file to parse. Currently the selection of these two components is quite manual--you need to create a "launch configuration", which is a JSON description of the debugging session.

Alternatively, to create the launch profile you may be able to

1. Select Run -> Open Configurations from the VS Code menubar. This will load a launch.json file into the editor. You may have existing configurations, or it may be empty.
2. Press Add Configuration... and select the Daffodil Debug - Launch option.

Once the launch.json has been created it will look something like this

{
  "type": "dfdl",
  "request": "launch",
  "name": "Ask for file name",
  "program": "${command:AskForProgramName}",
  "stopOnEntry": true,
  "data": "${command:AskForDataName}",
  "infosetOutput": {
    "type": "file",
    "path": "${workspaceFolder}/infoset.xml"
  },
  "debugServer": 4711
}

This default configuration will prompt you to select the schema and data files. You can also map the "program" and "data" elements to your files to avoid being prompted each time.

Note the use of ${workspaceFolder} for files in the VS Code workspace and the absolute paths for files outside of it.

{
  "type": "dfdl",
  "request": "launch",
  "name": "DFDL parse: My Data",
  "program": "${workspaceFolder}/schema.dfdl.xsd",
  "stopOnEntry": true,
  "data": "/path/to/my/data",
  "infosetOutput": {
    "type": "file",
    "path": "${workspaceFolder}/infoset.xml"
  },
  "debugServer": 4711
}

Launch a debugging session

Using the launch profile above you'd see a DFDL parse: My Data menu item at the top of the "Run and Debug" pane (Command-Shift-D). Then press the "play" button to start the debugging session.

In the Terminal you'll see log output from the debugger backend:

05:49:31.015 [io-compute-0] INFO org.apache.daffodil.debugger.dap.DAPodil - 
******************************************************
A DAP server for debugging Daffodil schema processors.

Build info:
  version: 0.0.10
  daffodilVersion: 3.1.0
  scalaVersion: 2.12.13
  sbtVersion: 1.5.1
******************************************************
05:49:31.018 [io-compute-0] INFO org.apache.daffodil.debugger.dap.DAPodil - launched with options listenPort: 4711, listenTimeout: 10 seconds
05:49:31.031 [io-compute-0] INFO org.apache.daffodil.debugger.dap.DAPodil - waiting at tcp://0.0.0.0:4711

Your schema file will also be loaded in VS Code and there should be a visible marking at the beginning where the debugger has paused upon entry to the debugging session. You can then control the debugger using the available VS Code debugger controls.

Other options for launching a debugging session

• Option 1:

  • Open up the schema file you wish to debug
  • From inside the file open the Command Palette (Mac = Command+Shift+P, Windows/Linux = Ctrl+Shift+P)
  • Once the command Palette is opened start typing Daffodil Debug:
    • Option 1 = Daffodil Debug: Debug File - This will allow for the user to fully step through the schema (WIP), once fully completed will produce a infoset to a file named SCHEMA-infoset.xml which it then opened as well.
    • Option 2 = Daffodil Debug: Run File - This will just run the schema through producing the infoset to a file named SCHEMA-infoset.xml.

• Option 2:

  • Open up the schema file you wish to debug
  • Click the play button in the top right, you will get two options:
    • Option 1 = Debug File - This will allow for the user to fully step through the schema (WIP), once fully completed will produce a infoset to a file named SCHEMA-infoset.xml which it then opened as well.
    • Option 2 = Run File - This will just run the schema through producing the infoset to a file named SCHEMA-infoset.xml which it then opened as well.

Custom views

Infoset tools

Find the infoset tools from the command menu (Mac = Command+Shift+P, Windows/Linux = Ctrl+Shift+P)

Inputstream Hex Viewer

Find the hex view from the command menu (Mac = Command+Shift+P, Windows/Linux = Ctrl+Shift+P)

Common errors and solutions

1. Wrong JDK. Be sure you're running Java 11.

On MacOS, using Homebrew:

brew install java11

Add change JAVA_HOME in your ~/.zshrc or equivalant:

# Java 11
export JAVA_HOME=/usr/local/Cellar/openjdk@11/11.0.12

Be sure code is in your PATH by following the instructions here.

With JAVA_HOME set to the Java 11 install, run code in your shell terminal.

Reporting problems and requesting new features

If problems are encountered or new features are desired, create tickets here.

Getting help

If you need help or guidance on using Daffodil and its tooling, please engage with our community on our mailing lists or review the archives.