Cloudera Docs

What's New in Apache Atlas

New features and functional updates for Atlas are introduced in Cloudera Runtime 7.3.2, its service packs, and cumulative hotfixes.

Cloudera Runtime 7.3.2.10000 SP1

Impala lineage uses the Impala operation type
Impala lineage events include an explicit operationType field. Apache Atlas reads that value when it builds Impala process and column lineage so the lineage graph reflects the operation without relying only on parsing the full queryText. For queries that insert into a table or overwrite a table, the process segment in lineage qualified names uses INSERT or INSERT_OVERWRITE instead of QUERY or QUERY_WITH_CLAUSE.
For more information, see Impala lineage and Impala entities created in Atlas.
Atlas UI enhancements: React dashboard, contextual search, Create menu, and glossary import

The React-based Apache Atlas user interface adds an operational dashboard, a context-aware global search bar, a unified + Create entry point, and a single Create Glossary workflow that toggles between manual creation and file import.

  • The Overview dashboard surfaces totals for entities and classifications, entity status distribution (Active, Shell, Deleted), classification types in use, entity type distribution, recently added entities, Kafka topic summary, and a Recent Activity stream for Created, Updated, Deleted, Purge, Import, and Export events.
  • The global header includes primary + Create actions for Entities, Classifications, Glossaries, Business Metadata, and Enums.
  • Quick search supports suggestions and a scope drop-down for Entities, Classifications, Terms (glossary), and Business Metadata.
  • In Create Glossary, you can download import templates and import glossary terms by dragging and dropping .csv, .xls, or .xlsx files.

You can still switch between Classic and New UI experiences. The New UI search panel lists entity types, classifications, and glossary terms with filtering for empty service types, unused classifications, and category or term glossary views, plus flat tree navigation for entities and classifications. For more information, see Apache Atlas dashboard tour, Using Basic Search, and Creating glossaries.

Distributed notification processing framework

Apache Atlas introduces a two-phase distributed processing framework to handle high-throughput metadata ingestion from sources like Spark and Hive. This moves Atlas from a single-threaded sequential ingestion pipeline to a parallel pipeline using Kafka topics. The preprocessor routes events to ATLAS_METADATA_* and ATLAS_LINEAGE_* topics, which are then processed concurrently by multiple consumers. This significantly improves horizontal scalability while preserving referential integrity by deterministically routing related entities to the same topic. For more information, see Configuring distributed notification processing.

TLS 1.3 support

Apache Atlas supports TLS 1.3 for stronger security and compliance. In Cloudera Manager, advanced TLS protocol and cipher options, including inheritance from global TLS settings, are available in the Atlas service configuration. Cloudera recommends upgrading to TLS 1.3 when possible. For more information, see Configuring TLS/SSL for Apache Atlas.

Header-based authentication

Apache Atlas supports header-based authentication so a trusted gateway can pass identity using HTTP headers. Default values align with x-awc-username, x-awc-roles, and x-awc-requestid. Configure atlas.authn.header.* keys in atlas-application.properties. For more information, see Configuring header-based authentication for Apache Atlas.

Optional PostgreSQL backend for JanusGraph and entity audit storage

Atlas can optionally use PostgreSQL as the JanusGraph KeyColumnValue persistence backend (atlas.graph.storage.backend=rdbms and related JDBC settings) to reduce footprint when you do not require a horizontally scalable HBase-backed graph store. You can also store entity audit events in a relational database by setting atlas.EntityAuditRepository.impl to org.apache.atlas.repository.audit.rdbms.RdbmsBasedAuditRepository. Default installations continue to use the existing HBase unless you select PostgreSQL during setup.

Cloudera Manager now provides a native Backend Selection step during the Atlas Add Service wizard to simplify configuring the PostgreSQL connection, including support for SSL/TLS-encrypted JDBC connections. For more information, see Configuring a PostgreSQL database for Atlas and Installing Atlas using Add Service.

Faster Atlas entity detail for large relationship graphs
The first load of an entity detail page omits large embedded relationship payloads so the page opens faster. On the Relationships tab, you can switch between Table mode (a card per relationship type with counts, sort, optional deleted relationships, and paged loads as you scroll) and Graph mode. The relationship search API supports pagination, optional sorting, and more reliable counts for very large graphs. Together, these changes address cases such as Apache Ozone buckets or wide Hive tables for which the detail view previously failed to display Properties, Classifications, Relationships, or Audits because the response was too large.
For more information, see Apache Atlas dashboard tour.

Cloudera Runtime 7.3.2.100 CHF 1

Faster Atlas entity detail for large relationship graphs
The first load of an entity detail page omits large embedded relationship payloads so the page opens faster. On the Relationships tab, you can switch between Table mode (a card per relationship type with counts, sort, optional deleted relationships, and paged loads as you scroll) and Graph mode. The relationship search API supports pagination, optional sorting, and more reliable counts for very large graphs. Together, these changes address cases such as Apache Ozone buckets or wide Hive tables where the detail view previously failed to show Properties, Classifications, Relationships, or Audits because the response was too large.
For more information, see Apache Atlas dashboard tour.
Impala lineage uses the Impala operation type
Impala lineage events include an explicit operationType field. Apache Atlas reads that value when it builds Impala process and column lineage so the lineage graph reflects the operation without relying only on parsing the full queryText. For queries that insert into a table or overwrite a table, the process segment in lineage qualified names uses INSERT or INSERT_OVERWRITE instead of QUERY or QUERY_WITH_CLAUSE.
For more information, see Impala lineage and Impala entities created in Atlas.

Cloudera Runtime 7.3.2:

Cloudera Runtime 7.3.2 introduces new features of Atlas and includes all service packs and cumulative hotfixes from 7.3.1.100 through 7.3.1.706. For a comprehensive record of all updates in Cloudera Runtime 7.3.1.x, see New Features.

New React-based user interface for Apache Atlas
Apache Atlas now features a redesigned React-based user interface (UI) that offers enhanced usability and streamlined metadata management. You can switch between the Classic and New UI experiences. The new interface introduces an improved search panel that automatically lists all available entity types, classifications, and glossary terms, with one-click access to relevant members. Enhanced filtering capabilities allow you to display empty service types, unused classifications, and toggle between category or term views in the glossary. Additionally, the interface displays entities and classifications in a collapsed flat tree view to simplify navigation of complex metadata hierarchies.
For more information, see Apache Atlas dashboard tour.
Apache Atlas component upgraded to 2.4.0

The Atlas runtime component is upgraded from 2.1.0 to 2.4.0. Several stability and correctness fixes are included from the upstream release for bugs, including UI improvements for classification propagation settings.

Atlas auto-purging introduced

The automated entity auto-purging feature addresses potential performance and storage issues caused by the manual purge strategy. Previously, Atlas preserved metadata by only marking entities as deleted (soft-delete). This led to query performance degradation and increased storage usage as soft-deleted entities accumulated. While these entities could be manually deleted by using the PUT /admin/purge/ API call, the process failed to remove column lineage entities associated with soft-deleted process entities.

The new, cron-based system can clean up obsolete process entities, including their column lineage entities, that are no longer relevant. This prevents sparse graphs and significantly improves metadata hygiene and query performance. For more information, see Atlas Auto-Purging overview.

Support replication of Atlas data from on-premises to on-premises

Atlas now supports asynchronous import of metadata using Kafka. Previously, the system relied on synchronous imports, which required the HTTP connection to remain open until the entire process completed. This caused timeouts for large datasets and making concurrent imports fragile. With asynchronous import, a client submits an import request that is immediately staged and queued as a Kafka message, and receives an import ID in response without waiting for processing to finish. Atlas processes the import in the background and persists the request state, including received time, processing start time, completion time, and outcome.

The following new REST API endpoints are available:
  • POST /api/atlas/admin/async/import — submit an asynchronous import; returns immediately with an import ID
  • GET /api/atlas/admin/async/import/status — list all async import statuses
  • GET /api/atlas/admin/async/import/status/{importId} — get the status of a specific import
  • DELETE /api/atlas/admin/async/import/{importId} — abort a specific queued import
Atlas upgraded to use JDK 17
Atlas now runs on Java 17, upgraded from Java 8. JDK 17 is a Long-Term Support (LTS) release that brings improved performance, enhanced security, and better long-term maintainability. JDK 17 has the following key benefits for Atlas users:
  • Improved garbage collection, resulting in lower latency and more efficient memory usage for metadata-intensive workloads.
  • Stronger cryptographic algorithms reducing security vulnerabilities.
  • Long-term support guaranteed until at least 2029, ensuring continued security patches.
Logback introduced as logging framework

Apache Atlas now uses Logback as its logging framework, replacing Log4j2. This change enhances security and simplifies log management. It also enables you to add any new properties overriding existing properties.

The new logging framework provides simplified configuration by using native XML configuration instead of the .properties file.
  • Go to Cloudera Manager > Atlas Server XML Override to replace the complete configuration file.

Configuration values for file size and rotation remain the same.