Why it matters

Installing the plugin is one command or one unzip; understanding what the install changed is what saves the next hour. Two install details decide whether a first session works:

  • the plugin is not just a jar — the assembly zip also updates lib/core/ so Jinja-based SQL resolves, and

  • the version pins are hard: the plugin requires Apache Hop 2.20 (development builds run 2.20.0-SNAPSHOT) and Java 21. Nothing else is supported.

This module installs the plugin by either supported route — Hop Marketplace or the assembly zip into $HOP_HOME — runs the first-time Configure EDW setup helper, and proves the local training database answers. That foundation is what modules 2 to 8 build on. Budget 30 minutes.

Concepts

The plugin in one picture

The plugin adds four model families to a Hop project — source (.hsm), Data Vault (.hdv), Business Vault (.hbv) and dimensional (.hdm) — plus the catalog and the run outputs around them. This module builds only the ground floor: the plugin itself, the shared project metadata, and a database that answers.

The Data Hopper EDW overview: sources

Where the plugin installs

The assembly zip is a complete plugin layout, not a single file. Unzipped into $HOP_HOME it writes two places:

$HOP_HOME/
  plugins/misc/hopper-edw/    plugin jar, icons, and the offline documentation (docs/)
  lib/core/                   Jinja (Jinjava) SQL support

Both lines matter. The folder under plugins/misc/hopper-edw/ is what Hop loads: after a restart, new metadata types appear under Metadata → EDW, the update and validation actions are available in workflows, and .hsm, .hdv, .hbv, .hdm and .hlv files open in Explorer tabs. Inside it, plugins/misc/hopper-edw/docs/ carries the offline guides — the same folder these training modules ship in. lib/core/ receives the Jinja SQL support; copying only the plugin jar is not enough for Jinja SQL.

What Configure EDW setup creates

A new Hop project is offered the standard catalog and the four shared configuration objects the first time you open a source, Data Vault, Business Vault, or dimensional model. Accept that first-time prompt, or run Tools → Configure EDW setup…​ from the menu if you dismissed it — the same dialog appears either way, and it creates, when missing:

  • the Data catalog connection local-catalog — a FILE catalog with default storage ${PROJECT_HOME}/catalog-data (retail-example uses a gitignored work/edw-catalog/ instead), and

  • the shared model configurations source-model, data-vault, business-vault and dimensional — the objects each model points at with Configuration on the Edit model dialog.

It never recreates what exists: elements already present stay selected and disabled in the dialog, and only missing ones can be created. And it deliberately does not create a resource definition group — you create that yourself once the first .hdv checks clean (module 4).

Plugin-wide options

The per-model settings have a global counterpart. Hop GUI keeps the plugin-wide Hopper EDW options under Configuration → Hopper EDW — for example whether hash key columns are drawn in the model, and the default-on Enforce Unicode-capable target databases (EDW) check. These settings live in the Hop GUI configuration, not inside the model files.

Hopper EDW options in the Hop configuration perspective

Guided walkthrough

Install the plugin

Route 1 — Hop Marketplace (needs Hop 2.20.0-SNAPSHOT). Import the shareable repository definition, install the plugin, then restart Hop:

./hop marketplace repo import \
  https://raw.githubusercontent.com/ProjectDataHopper/hopper-edw/refs/heads/main/hop-marketplace-repo.yaml
./hop marketplace install hopper-edw

In Hop GUI the same route runs from Tools → Marketplace…: import the repository on the Repositories tab, then install from the Plugins tab.

Route 2 — assembly zip (works offline). Take hopper-edw-<version>.zip from a release, or build your own with mvn clean package, and unzip it into $HOP_HOME. The zip updates both plugins/misc/hopper-edw/ and lib/core/; do not copy only the plugin jar out of it, because Jinja SQL then fails. If you ever installed 0.9.0 as hop-datavault, remove $HOP_HOME/plugins/misc/datavault first so the old and new folders do not stack.

Restart Hop after either route so the plugin registry reloads. The install is complete when Metadata → EDW shows the EDW metadata types and a .hdv file opens in an Explorer tab.

Open the training project

  1. Start Hop GUI and open the project folder retail-example/ (File → Open project).

  2. Confirm the connections CRM, Vault and OPS exist in Metadata → Relational database connection. On retail-example they ship with the project metadata: CRM is the source, Vault is the warehouse, OPS is the operations database. On your own project, these three are the first connections you create.

  3. From the repository root, start the local database once:

    ./scripts/run-postgres.sh up

  4. Check the success line — it names the endpoint and the extra databases: PostgreSQL is ready (test@localhost:54320/test; also test_source, test_edw, test_ops, test_ops_analytics for retail-example; …​). Remember port 54320 and user/password/database test/test/test; the container is hop-data-vault-postgres-local.

  5. Optional, and only needed for module 7 (lineage, quality and operations): start the local Marquez stack the same way. It holds the lineage structure that a Hop Lineage View reads, while load durations keep coming from the OPS connection.

    ./scripts/run-marquez.sh up

  6. Check its success lines — Marquez is ready. followed by API: http://localhost:5001/api/v1/lineage and UI: http://localhost:3001. The containers are hop-marquez-api, hop-marquez-web and hop-marquez-db; ./scripts/run-marquez.sh status reports whether the API answers, down stops the stack and reset wipes its database too.

Configure EDW setup

  1. Open Tools → Configure EDW setup…​ and read the dialog. On retail-example everything the dialog offers already exists, so its elements appear selected and disabled; on a fresh project the same dialog creates the FILE catalog connection local-catalog and the shared configurations.

    Configure EDW setup — standard catalog and model-configuration elements

  2. Open the toolbar EDW button (or Help → EDW documentation) and confirm the offline documentation opens — the guides and these training modules ship inside the plugin at plugins/misc/hopper-edw/docs/ and load with no network.

  3. Optional, only on a fresh project: create an empty source model or Data Vault model file (Explorer → New) so the first-time prompt can appear; if you dismissed it, reopen Tools → Configure EDW setup…​ from the menu. Uncheck Do not show this message again if you want the automatic prompt back.

Try it yourself

Run the training environment end to end, from the repository root:

# 1. local PostgreSQL (container hop-data-vault-postgres-local, port 54320)
./scripts/run-postgres.sh up

# 2. the finished retail warehouse, so you can compare with your own work later
./scripts/run-hop.sh retail-example workflows/run-retail-initial.hwf

# 3. optional, for module 7: the local Marquez stack a Hop Lineage View reads
./scripts/run-marquez.sh up

The first run-hop.sh call builds or reuses the local Hop image (docker-hop:latest); if you rebuilt the plugin and want the image refreshed, force it with ./scripts/rebuild-hop.sh. The third command answers with Marquez is ready. and the API/UI URLs; skip it if you are not going to do module 7, and stop the stack later with ./scripts/run-marquez.sh down.

Check the warehouse the way the later modules do — in Hop GUI:

  1. Open the Database perspective (the database icon in the perspective bar).

  2. Select connection Vault and press Connect: schemas and tables load, so expanding Vault → public lists the warehouse tables — empty before the first load, or hub_*, sat_*, d_* and f_* after it. The other retail-example connections are CRM (source tables), OPS (the operations database) and OpsAnalytics.

  3. Press Open SQL editor on Vault and run a query with Run (Ctrl+Enter):

    select count(*) from f_orders;

    relation "f_orders" does not exist before the initial load is expected: that is the state a hand-built warehouse starts from in modules 2 to 4.

Checkpoint — you can show all five:

  • retail-example opens in Hop GUI,

  • Tools → Configure EDW setup…​ opens and shows local-catalog and the four shared configurations,

  • connection Vault connects in the Database perspective and lists the warehouse tables under public,

  • a query in Vault's SQL editor returns a result (after the initial load), and

  • ./scripts/run-marquez.sh status reports the API accepting requests on http://localhost:5001.

Questionnaire

Answer the seven questions. Every answer comes with its reasoning, and Copy my results gives you a text summary to keep.

1. Which Apache Hop version does this plugin require?

2. Which JDK is the required one?

3. What does Configure EDW setup create, when the elements are missing?

4. What does Configure EDW setup deliberately NOT create?

5. After the install, where does the plugin live inside $HOP_HOME?

6. Scenario: a colleague copied only the plugin jar out of the assembly zip into their Hop installation. Hop starts, but pipelines that generate Jinja-based SQL fail. What is the missing piece?

7. The local training database: which endpoint and credentials does run-postgres.sh up prepare?

Where to go next

← Back to the training index