Unity Catalog objects and storage

DP-750 · Set up and configure an Azure Databricks environment · Updated

Unity Catalog is the central governance layer of Azure Databricks. Knowing its object model and where data is physically stored is the base for most other questions.

Object hierarchy

Everything is addressed with a three-level name: catalog.schema.object.

  • Metastore: top-level container, one per region, attached to workspaces. It holds metadata and permissions.
  • Catalog: first level, often per environment or domain (dev, prod).
  • Schema (database): second level, groups tables, views, volumes and functions.
  • Objects: tables, views, materialized views, volumes, functions and models.

Managed and external storage

Managed tableExternal table
Data locationUnity Catalog managed storageA path you control
DROP TABLEDeletes metadata and dataDeletes metadata only
FormatDelta (default)Delta and other formats

Managed storage is resolved from the lowest level that defines it: schema, then catalog, then metastore. Set it with MANAGED LOCATION:

CREATE SCHEMA catalog1.schema2
MANAGED LOCATION 'abfss://data@account.dfs.core.windows.net/data';

LOCATION is used for external tables, not for schemas or catalogs.

Connecting to ADLS Gen2

Unity Catalog reaches storage through two securables:

  1. A storage credential wraps an Azure managed identity (through an Access Connector for Azure Databricks) or a service principal that has access to the storage account.
  2. An external location pairs a storage path with that credential.

Users never get the storage keys. They are granted READ FILES or WRITE FILES on the external location, or privileges on the table.

Volumes

A volume is a governed folder for files, addressed as /Volumes/catalog/schema/volume/path. Managed volumes live in managed storage and external volumes point to an external location. Use them for raw landing files, libraries and ML artifacts.

Useful commands

CREATE CATALOG IF NOT EXISTS prod;
CREATE SCHEMA prod.sales;
CREATE VOLUME prod.sales.landing;

Exam traps

  • A schema needs MANAGED LOCATION, not LOCATION.
  • A managed location path must sit under an external location that exists.
  • The old Hive metastore (hive_metastore) is not governed by Unity Catalog.
  • Dropping an external table does not delete files. Dropping a managed table does.
  • A storage credential is not the same as an external location.

Practice questions

Choose an answer, then review the explanation to check your understanding.

1. You must create schema2 in catalog1 so that its managed tables are stored in a specific ADLS Gen2 path. Which statement is correct?

Show answer

Answer: B.MANAGED LOCATION sets where managed tables of the schema are stored. The path must be covered by an external location.

2. How many Unity Catalog metastores can be attached to a workspace?

Show answer

Answer: A.A workspace is attached to one metastore. A metastore is per region and can serve many workspaces.

3. You drop an external table. What happens to the data files?

Show answer

Answer: B.Dropping an external table removes only metadata. A managed table is the one whose data is deleted.

4. You need governed access to arbitrary files such as images and CSVs, not tables. What do you create?

Show answer

Answer: A.Volumes govern non-tabular files under catalog.schema.volume.

DP-750 course outline

All 8 lessons · Course overview
  1. Choosing Azure Databricks computeSet up and configure an Azure Databricks environment Free
  2. Unity Catalog objects and storageSet up and configure an Azure Databricks environment Free
  3. Delta Lake tables and maintenanceIngest and transform data Full access
  4. Auto Loader and Structured StreamingIngest and transform data Full access
  5. Lakeflow Declarative PipelinesBuild and manage data pipelines Full access
  6. Lakeflow Jobs, scheduling and deploymentBuild and manage data pipelines Full access
  7. Securing data with Unity CatalogSecure, govern and optimize workloads Full access
  8. Lineage, audit logs and data quality monitoringSecure, govern and optimize workloads Full access