Fixed issues in Ozone 718.1.0 parcel

You can review the list of reported issues and their fixes in Ozone 718.1.0 parcel. Fixed issues represent selected issues that were previously logged through Cloudera Support, but are now addressed in the current Ozone parcel release. These issues may have been reported in previous versions of Runtime as a known issue; meaning they were reported by customers or identified by Cloudera Quality Engineering teams.

TSB (2024-748) - Potential Ozone data loss in CDP Private Cloud Base 7.1.8 caused by container delete
CDPD-50524: HDDS-8142: Check if no entries in Block DB for a container on container delete
CDPD-50524: HDDS-8118: Fail container delete on non empty chunks dir
HDDS-8108: gRPC channel created for replication client not shutdown properly
HDDS-8129: ContainerStateMachine allows two different tasks with the same container id running in parallel
HDDS-8019: IllegalStateException: call already closed at GrpcXceiverService.onCompleted
CDPD-49916: HDDS-7156: Reset pending delete block count
CDPD-48932: HDDS-8044: Ensure GrpcOutputStream is closed
CDPD-49916: HDDS-7091: Filter containers with pending deletion blocks before checking policy
CDPD-49916: HDDS-7126: Clean deletedBlock records of deleted containers
CDPD-49916: HDDS-7306: DeleteBlock retry count increased at incorrect place
CDPD-49918: HDDS-8037: Improve logging in EC Reconstruction putBlock precondition check
HDDS-8020: File checksum helper leaking client
HDDS-7931: EC: ManagedChannelImpl not cleaned up properly
HDDS-7829: SCM to reject adding container to closed pipeline
HDDS-7738: SCM terminates when adding container to a closed pipeline
HDDS-7920: User cannot list their own volumes without access to root volume
HDDS-7915: Force close QUASI_CLOSED replicas only when the container is CLOSED in Legacy RM
HDDS-7923: [EC] Reconstruction is failing with IndexOutOfBoundsException
HDDS-7838: gRPC channel created block input/output stream not shutdown properly
HDDS-7455: ClassCastException: OzoneTokenIdentifier cannot be cast to String
CDPD-49042: Cherry pick HDDS-7859
CDPD-47991: Ozone - Upgrade Netty to 4.1.86.Final due to CVE-2022-41881, CVE-2022-41915
HDDS-7801: Bucket not found when calling getKeyInfo with tenant context
HDDS-7755: Ensure that acquired locks are always released
HDDS-7787: GetChecksum for EC files can fail intermittently with IndexOutOfBounds exception
HDDS-7761: EC: ReplicationManager - Use placementPolicy.replicasToRemoveToFixOverreplication in EC Over replication handler
HDDS-7775: EC: Exception encountered while deleting UNHEALTHY replica in Datanode
HDDS-6447: Refine SCM handling of unhealthy container replicas
HDDS-7753: Simplify DatanodeDetails#toString to improve log messages
HDDS-7560: Placement Policy Interface changes to handle Overreplication
CDPD-48225: HDDS-7754: Download of container is failing with SSL/TLS error during re-replication
HDDS-7726: EC: Enhance datanode reconstruction log message
HDDS-7739: EC: Increase the information in the RM sending command log message
HDDS-7695: EC metrics related to replication commands don't add up
HDDS-7729: EC: ECContainerReplicaCount should handle pending delete of unhealthy replicas
HDDS-7727: EC: SCM unregistered event handler for DatanodeCommandCountUpdated
HDDS-7666: EC: Unrecoverable EC containers with some remaining replicas may block decommissioning
HDDS-7696: MisReplicationHandler does not consider QUASI_CLOSED replicas as sources
HDDS-7661: Ratis Misreplication Handler
HDDS-7692: Remove usage of MonotonicClock and use default system clock instead
HDDS-7463: SCM Pipeline scrubber never able to cleanup allocated pipeline
HDDS-7683: EC: ReplicationManager - UnderRep maintenance handler should not request nodes if none needed
HDDS-7635: Update failure metrics when allocate block fails in preExecute
HDDS-7654 EC: ReplicationManager - merge mis-rep queue into under replicated queue
HDDS-7621: Update SCM term in datanode from heartbeat without any commands
HDDS-7605: Improve logging in Container Balancer
HDDS-7616: EC: Refactor Unhealthy Replicated Processor
HDDS-7426: Add a new acceptance test for Streaming Pipeline
HDDS-7478: [Ozone-Streaming] NPE in when creating a file with o3fs
HDDS-7425: Add documentation for the new Streaming Pipeline feature
HDDS-7438: [Ozone-Streaming] Add a createStreamKey method to OzoneBucket
HDDS-7431: [Ozone-Streaming] Disable data steam by default
HDDS-6955: [Ozone-streaming] Add explicit stream flag in ozone shell
HDDS-6867: [Ozone-Streaming] PutKeyHandler should not use streaming to put EC key
HDDS-6842: [Ozone-Streaming] Reduce the number of watch requests in StreamCommitWatcher
HDDS-6592: [Ozone-Streaming] Fix ContainerStateMachine#applyTransaction assert error
HDDS-5666: Add option to createKey via streaming api in Freon
HDDS-6500: [Ozone-Streaming] Buffer the PutBlockRequest at the end of the stream
HDDS-6137: [Ozone-Streaming] Refactor KeyDataStreamOutput
HDDS-5798: [Ozone-Streaming] Setup TlsConf parameters
HDDS-6388: [Ozone-Streaming] Streaming write support both pipeline model and star model
HDDS-6355: [Ozone-Streaming] Fix CheckStyle problem
HDDS-6229: [Ozone-Streaming] Data Channel abstraction on datanode
HDDS-6282: Fix BlockDataStreamOutput#doFlushIfNeeded NPE
HDDS-5487: [Ozone-Streaming] BlockDataStreamOutput support FlushDelay
HDDS-6298: Add XceiverServerRatis stream config
HDDS-6138:[Ozone-Streaming] Define a limit on the size of the retry bufferList
HDDS-6178: [Ozone-Streaming] Fix NPE in HDDS-6139
HDDS-6139: [Ozone-Streaming] Fix incorrect computation of totalAckDataLength
HDDS-6130: [Ozone-Streaming] When releaseBuffers will get “Couldn 't find the required future
HDDS-6039: Define a minimum packet size during streaming writes
HDDS-5851: [Ozone-Streaming] Define a PutBlock/maxBuffer fixed boundary for streaming writes
HDDS-5743: [Ozone-Streaming] Add option to write files via streaming api in ofs and o3fs
HDDS-5879: [Ozone-Streaming] OzoneBucket add the createMultipartStreamKey method
HDDS-5961: [Ozone-Streaming] update the usage space of Containers in the stream write
HDDS-5987: [Ozone-Streaming] Add XceiverClientRatis stream config
HDDS-5763: Provide an Executor for each LocalStream in ContainerStateMachine
HDDS-5895: [Ozone-Streaming] Make raft.server.data-stream.client.pool.size configurable
HDDS-5674: [Ozone-Streaming] Handle client retries on exception
HDDS-5849: [Ozone-Streaming]Write exceptions occur after checksum is enabled
HDDS-5486: [Ozone-Streaming] Streaming supports writing in Pipline mode
HDDS-5742: Avoid unnecessary Bytebuffer conversions
HDDS-5705: [Ozone-Streaming] Change ByteBufStreamOutput to ByteBufferStreamOutput
HDDS-5599: [Ozone-Streaming]drop BufferPool and ChunkBuffer to avoid buffer copying
HDDS-5488: [Ozone-Streaming] Add a new BlockOutputStream/KeyOutputStream to support streaming api
HDDS-5480: [Ozone-Streaming] Client and server should support stream setup
HDDS-5481: Fix stream() and link() method in ContainerStateMachine
HDDS-5452: Add link method to ContainerStateMachine for Ratis streaming
HDDS-5366: [Ozone-Streaming] Implement stream method to ContainerStateMachine
HDDS-7371: Create properties for all dependency versions
HDDS-7620: Check term for async commands before processing
HDDS-7521: EC: Misreplication Handler changes for Placement Policy interface changes
HDDS-7636: Remove hadoop security dependency in org.apache.hadoop.HDDS.security.ssl package
HDDS-7628: Disable flaky TestOzoneContainerWithTLS#testCreateOzoneContainer
HDDS-7642: EC: Add debug logging to the Replication Manager check handlers
HDDS-7592: Recursive delete of directory multiple time impacts some directory not removed
HDDS-7627: EC: Bug fix for calculating Misreplication Count
HDDS-7643: rocksdb iterator is not closed properly
HDDS-7503: CLI admin namespace -quota should return whole file system disk capacity consumed by all replicas
HDDS-7582: ECUnderReplicationHandler does not consider pending adds when finding targets
HDDS-7640: EC: UNHEALTHY replicas not replaced by healthy replicas from a CLOSED container by RM
HDDS-7008: Mark TestOzoneHARatisLogParser as flaky
HDDS-7619: Update SCM term in datanode when command is received
HDDS-7591: Add a handler for over replicated Ratis containers to RM
HDDS-7628: Disable flaky TestOzoneContainerWithTLS#testContainerDownload
HDDS-7618: Replication Commands should timeout if not processed on datanodes in time
HDDS-7595: Add OmId in OM's UI
HDDS-7589: Unify multipart stream reading
HDDS-7614: Add subscription mechanism to ContainerReplicaPendingOps
HDDS-7580: Add option to show key count in DBScanner
HDDS-7572: Use keyManager and trustManager provided by keyStoreFactory in datanode grpc services
HDDS-7617: Mark testECContainerRecoveryWithTimedOutRecovery as flaky
HDDS-7587: Update NOTICE and LICENSE in license/bin/
HDDS-7603: Enable Erasure coding and encryption flag on FileStatus for ofs
HDDS-7265: ScatterRackPolicy for Ratis pipeline provider container placement
HDDS-7502: Recon: Show number of open containers on the Overview page
HDDS-7615: Import Guava classes directly, not from hadoop-thirdparty
HDDS-7492: Extend Placement Policy Interface to select mis-replicated replicas to copy
HDDS-7597: Clarify SCM admin access denied message
HDDS-7464: Container Report at SCM is not coming separately for ICR and FCR in prometheus endpoint
HDDS-7550: Recon: UI sorting not functioning as expected
HDDS-7599: Bump commons-net from 3.6 to 3.9.0
HDDS-7526: Avoid overwriting replication config on existing bucket when quota is set
HDDS-7472: EC: Fix NSSummaryEndpoint#getDiskUsage for EC keys
HDDS-5704: Ozone URI syntax description in help content needs to mention about ozone service id
HDDS-7555: Upgrade Ratis to 2.4.2-8b8bdda-SNAPSHOT
HDDS-7541: FSO recursive delete directory with hierarchy takes much time for cleanup
HDDS-7581: Fix update-jar-report for snapshot
HDDS-7253: Fix exception when '/' in key name
HDDS-7579: Use Netty 4.1.77 for consistency
HDDS-7562: Suppress warning about long filenames in tar
HDDS-7563: Add a handler for under replicated Ratis containers in RM
HDDS-7497: Fix mkdir does not update bucket's usedNamespace
HDDS-7567: Invalid entries in LICENSE
HDDS-7575: Correct showing of RATIS-THREE icon in Recon UI
HDDS-7540: Let reusable workflow inherit secrets
HDDS-7568: Bump copyright year in NOTICE
HDDS-7394: OM RPC FairCallQueue decay decision metrics list caller username in the metric
HDDS-7510: Recon: Return number of open containers in `/clusterState` endpoint
HDDS-7561: Improve setquota, clrquota CLI usage
HDDS-6615: EC: Improve write performance by pipelining encode and flush
HDDS-7554: Recon UI should show DORMANT in pipeline status filter
HDDS-7540: Separate scheduled CI from push/PR workflows
HDDS-7500: Ozone recon server Memory Optimization
HDDS-7536: Remove unused dependency declarations
HDDS-7486: Support KeyStoreFactory which supports keyManager and trustManager reload
HDDS-7289: Bump protobuf-java from 3.19.2 to 3.19.6
HDDS-7535: Eliminate duplicated config in LegacyReplicationManager
HDDS-7449: Avoid overwriting bucket encryption properties when quota/replication config are set on an existing bucket
HDDS-7538: Remove unused OmUtils#getMD5Digest
HDDS-7534: Intermittent failure in TestKeyDeletingService#checkDeletionForEmptyKey
HDDS-7353: Cleanup of unused code in SCM
HDDS-7481: Set quota with reduced size than actual used do not fail
HDDS-7187: EC: Retry failed writes before rewrite to a new block group
HDDS-7532: EC: ReplicationManager - remove calls to ECHealthCheck from under and over replication processing
HDDS-7528: EC: ReplicationManager - refactor logic to send datanode commands into a central place
HDDS-7092: EC: Offline Recovery with simultaneous Over Replication and Under Replication
HDDS-7462: EC: Fix Reconstruction Issue with StaleRecoveringContainerScrubbingService
HDDS-7518: Fix intermittent failure in ozonesecure replication test
HDDS-5526: ContainerBalancer#checkConditionsForBalancing pre-emptively checks iteration limits
HDDS-7280: New test addition to TestOMBucketCreateRequestWithFSO
HDDS-7476: SCM root CA stored under HDDS.datanode.dir if ozone.metadata.dirs is not specified
HDDS-4565: [FSO]Delete : quotaReleased on directory deletion should consider all sub paths
HDDS-7498: Add permission check when `--user` is specified in `ozone sh volume list`
HDDS-7346: Cannot set bucket args when the bucket has quota set
HDDS-7470: [Quota] bucket is created crossing quota makes OM crash and unable to start
HDDS-7260: dfs -du should return correct disk space consumed by all replicas
HDDS-7474: Recon: Add count to containers page tab label
HDDS-7245: Recon: Add button to trigger OM DB Sync (Frontend)
HDDS-4402: Recon dashboard page does not load until missing containers API returns data
HDDS-7488: EC: ReplicationManager: Move Mis-Replicated into a separate unhealthy state
HDDS-7487: Support fine grained certificate lifetime for efficient test
HDDS-7095: allow on demand scanning for containers
HDDS-6971: EC: Add EC block checksum computer
HDDS-7101: EC: ReplicationManager - handle UNHEALTHY replicas
HDDS-7255: Add metrics for container reports events
HDDS-7490: fix metaTable error
HDDS-7468: Recon: clusterState API Changes for adding missingContainerCount
HDDS-7485: Bump commons-codec to 1.15
HDDS-7475: Remove outdated Hadoop dependencies
HDDS-7471: EC: Notify ReplicationManager when a heartbeat updates datanode command counts
HDDS-7446: Cleanup possibly onDisk unreferenced blocks
HDDS-7469: Recon: Add more information to Open Containers column on on the Datanodes page
HDDS-4385: Search function using container ID on Recon Missing Container
HDDS-7408: Recon: Refactor container status page
HDDS-7456: Recon: Refreshed at not working on Pipeline and DataNode page
HDDS-7323: Recon: Auto refresh toggle is switched back when visiting new site
HDDS-7360: Investigate TestRootedOzoneFileSystem test cases
HDDS-7314: Recon graceful shutdown while closing RocksDB handle
HDDS-7440: Delete package-info class for org.apache.hadoop.io.erasurecode.rawcoder to avoid conflict with Hadoop
HDDS-7453: Check certificate expiration at service startup, renew if necessary
HDDS-2642: Expose decommission / maintenance metrics via JMX
HDDS-5112: Recon Insights page does not list buckets when only one volume is present
HDDS-6896: EC: ReplicationManager - Add relevant metrics to the various ReplicationManager classes
HDDS-7326: Intermittent timeout in TestECContainerRecovery.testContainerRecoveryOverReplicationProcessing
HDDS-7451: Add UT of SyncDataFromOM for SequenceNumberNotFoundException
Upgrade RocksDB to 7.7.3
HDDS-7460: Bump snakeyaml from 1.32 to 1.33
HDDS-3419: Throw exception with correct code when available data nodes are not sufficient
HDDS-7362: Bring back the mechanism to share LRU cache between RocksDB in datanodes
HDDS-7419: Integrate the GetKeyInfo API to OFS
HDDS-7324: S3G: Avoid calling getS3VolumeContext by using GetGeyInfo API
HDDS-5866: Discrepancy in Trash directory in ofs vs o3fs
HDDS-7423: DB Scanner should allow iteration from specific key
HDDS-6757: Test "size.leaving.source.max" limit in ContainerBalancer
HDDS-7090: EC: delete empty closed EC container
HDDS-7025: Add cache metrics in OM
HDDS-7435: ContainerOperationClient cleanup
HDDS-6926: Add support for shaded protobufs used by hadoop-client/spark
HDDS-7447: Leak of TableIterator in DirectoryDeletingService
HDDS-7320: Enable Container SchemaV3 by default
HDDS-7429: Remove Tag from SCMMetadataStoreMetrics
HDDS-7363: Changes to Ozone CLI to validate if the jars in classpath files are present on an install
HDDS-7290: provide a config to increase the list batch size in OzoneFileSystem
HDDS-7244: Fix multiple reports queued up from same DN and using up heap
HDDS-7146: Recon: Add button to trigger OM DB Sync (Backend)
HDDS-7231: Integrate the GetKeyInfo API to key read flows
HDDS-7384: EC: ReplicationManager - implement deleting container handler
HDDS-7361: Add general metrics for queues in Datanode
HDDS-7432: Move command summary into Commands object in CommandQueue
HDDS-7420: Bump Spring framework from 5.2.20 to 5.3.23
HDDS-7421: Respect OZONE_LOGLEVEL and OZONE_ROOT_LOGGER for CLI commands
HDDS-7316: Print stacktrace to identify the location of RocksObject leaks
HDDS-1157: TestOzoneContainerWithTLS is failing with SSLHandshakeException
HDDS-7406: Remove unused the exception and improve debug log in KeyDeletingService
HDDS-7328: Improve Deletion of FSO Paths
HDDS-7370: Add pending commands in SCM to Datanode command count
HDDS-7220: SCM should use sub-ca certificate for token signature without HA enabled
HDDS-7424: Bump jetty to 9.4.49.v20220914
HDDS-7407: EC: Block allocation should not be stripped across the EC group
HDDS-7402: Adapt CommandQueue to track the count of each queued command type
HDDS-7422: Bump woodstox-core from 5.0.3 to 5.4.0
HDDS-7349: Flaky integration test have memory leak for RatisDropwizardExports
HDDS-7342: Move encryption-related code from MultipartCryptoKeyInputStream to OzoneCryptoInputStream
HDDS-7413: Fix logging while marking container state unhealthy
Revert HDDS-7253: Fix exception when '/' in key name
HDDS-7396: Force close non-RATIS containers in ReplicationManager
HDDS-7121: Support namespace summaries (du, dist and counts) for legacy FS buckets
HDDS-7258: Cleanup the allocated but uncommitted blocks
HDDS-7381: Cleanup of VolumeManagerImpl
HDDS-7253: Fix exception when '/' in key name
HDDS-7182: Add property to control RocksDB max open files
HDDS-7284: JVM crash for rocksdb for read/write after close
HDDS-7368: [Multi-Tenant] Add Volume Existence check in preExecute for OMTenantCreateRequest
HDDS-7403: README Security Improvement
HDDS-7199: Implement new mix workload Read/Write Freon command
HDDS-7248: Recon: Expand the container status page to show all unhealthy container states
HDDS-7141: Recon: Improve Disk Usage Page
HDDS-7369: Fix wrong order of command arguments in Nonrolling-Upgrade.md
HDDS-6210: EC: Add EC metrics
HDDS-7355: non-primordial scm fail to get signed cert from primordial SCM when converting an unsecure cluster to secure
HDDS-7356: Update SCM-HA.zh.md to match the English version
HDDS-6930: SCM,OM,RECON should not print ERROR and exit with code 1 on successful shutdown
Revert HDDS-7199: Implement new mix workload Read/Write Freon command which meets specific test requirements
HDDS-7354: SchemaV3 blockData not deleted in table
HDDS-7199: Implement new mix workload Read/Write Freon command which meets specific test requirements
HDDS-7351: Use jackson-bom to ensure consistent Jackson version
HDDS-7305: Fix Hadoop imports
HDDS-7341: EC: Close pipelines with unregistered nodes
HDDS-7058: EC: ReplicationManager - Implement ratis container replication check handler
HDDS-7352: OM log flooded by AWSV4AuthValidator
HDDS-7149: Update ratis version to 2.4.0 and thirdparty version to 1.0.2.
HDDS-7254: Document that moving SCM from non-HA to HA is currently unsupported
HDDS-6893: EC: ReplicationManager - move the empty container handling into RM from Legacy
HDDS-7340: Bump jackson-databind to 2.13.4.2
HDDS-7214: Continuous start and stop can have hanging threads in stopping
HDDS-7343: Do not log exception on file not found in getFileStatus()
HDDS-7337: Replace set-output in Github Actions workflow
HDDS-7261: Add container location cache metrics
HDDS-7330: Cleanup of BucketManagerImpl
HDDS-7303: EC: ECBlockReconstructedStripeInputStream should set initialized only at the end of init()
HDDS-7269: Remove fair lock configuration for container state management flows
HDDS-7300: Race condition between full data scan and block deletion
HDDS-7322: Remove unused dependencies hsqldb and jettison
HDDS-7308: Fix config example syntax highlight in OM HA
HDDS-7270: Fix bug in checking healthy replica placement policy check in EC underReplication handler
HDDS-7291: Fixing exception handling in case of non positive replica index
HDDS-7308: Removed cluster2 from serviceIds
HDDS-7315: SCM order of close of dbstore and other services
HDDS-7271: Ozone Integration test shows memory leak (graceful shutdown cleanup)
HDDS-6946: Bump Hadoop to 3.3.4
HDDS-7313: Update github actions for Node16
HDDS-7088: OM incorrectly detects SCM Ratis Group ID when OM and SCM are colocated with same Ratis storage directory
HDDS-6440: Handle custom metadata (x-amz-meta) during put-object through S3 API
HDDS-7236: Enable Recon SCM DB bootstrap by default
Revert HDDS-7271: Ozone Integration test shows memory leak (graceful shutdown cleanup)
HDDS-7271: Ozone Integration test shows memory leak (graceful shutdown cleanup)
HDDS-7304: EC: EC Decode can fail when byteBuffer from elastic pool is larger than chunksize
HDDS-7230: Implement GetKeyInfo API
HDDS-7299: Migrate simple tests in HDDS-container-service to JUnit5
HDDS-6664: Implements getUri in TrashOzoneFileSystem
HDDS-7287: Send deleteBlocksRequest with correct retry count
HDDS-7302: Fix inaccurate container block count caused by missing block
HDDS-7294: Move ozone manager background services to a separate package
HDDS-7286: Clean up ContainerManager
HDDS-7293: Bump jackson2 to 2.13.4
HDDS-7259: Fix uncounted blocksDeleted in BlockDeletingService
HDDS-7208: Erasure coding and encryption are not flagged on FileStatus
HDDS-7217: OM logs wrong bucket layout
HDDS-7211: Remove dead code from KeyManagerImpl
HDDS-7282: Add EstimatedKeyCount metrics for SCM DB
HDDS-7250: Add HttpServer metrics
HDDS-7219: Remove not applicable info from ozone bucket list for links
HDDS-7285: Fix concurrency issue in DeleteBlocksCommandHandler
HDDS-7235: EOFException occurs when executing TPC-DS using o3fs
HDDS-7283: Set coreSize equal to maxSize for threadPoolExecutor
HDDS-7268: EC: Fix tests for HealthCheck handlers of RM that use Replica Indexes for Ratis Containers
HDDS-7229: Introduce container location cache in ScmClient
HDDS-7206: Change the placement policy interface to allow existing nodes to be specified for Rack Scatter Policy
HDDS-7263: Add a handler for Quasi Closed containers to RM
HDDS-7232: Introduce container-level lock while handling container report events by SCM and Recon
HDDS-7222: Recon Inactive pipelines not shown
HDDS-6863: Add Group-Id and Ratis-Roles Information for OM UI
HDDS-7257: Bump snakeyaml to 1.32
HDDS-7160: Recon: Make Container and Bucket stats clickable in Overviews page
HDDS-6492: Add metric for failed container moves
HDDS-7145: Recon: Show last OM DB sync time on Overview page
HDDS-7039: EC: Handle the placement policy check in ECUnderReplicationHandler
HDDS-7075: Implement FileSystem listStatusIterator to support file list paging
HDDS-7251: Replace Log4j 1.x with Reload4j
HDDS-7240: List all volume operation should go through ACL check as well in order to trigger audit logging
HDDS-7239: EC: Add a Handler for CLOSING containers in Replication Manager
HDDS-7203: LookupKey Latency breakdown
HDDS-7185: Encode asterisk for parsing signature
HDDS-7221: EC: ReplicationManager - Encapsulate the under and over rep queues into a queue object
HDDS-7207: Fixed typo "layload" in om-echo command description as payload
HDDS-7225: Add Ratis tests for HealthCheck handlers of Replication Manager
HDDS-7144: Recon: Make only hostname fixed in Datanodes page
HDDS-7193: S3gateway Grpc Ranger Authorizer NPE Unhandled Exception
HDDS-7114: CHAINED and STAND_ALONE replication-type should be removed from help message
HDDS-7192: EC: ReplicationManager - create handlers to perform various container checks
HDDS-7209: Bump Jersey2 to 2.34
HDDS-7204: EC: Schedule UnderReplicatedProcessor and OverReplicatedProcessor threads in RM instead of StorageContainerManager
HDDS-7174: Migrate SCM integration tests to JUnit 5
HDDS-2476: Share more code between metadata and data scanners
HDDS-7086: DataNode UI to show failed volumes too
HDDS-7191: Separate prop for s3 admin
HDDS-7143: Recon: Fix left to right scrolling on Pipelines page
HDDS-7201: testContainerIsReplicatedWhenAllNodesGotoMaintenance is failing frequently
HDDS-7197: Schema V3 RocksDB instance statitics register as metrics collision
HDDS-7200: Fixed inaccurate get request status code
HDDS-7171: Add SupportedSourceVersion in annotation processors
HDDS-7125: Inaccurate numBlocksDeletion when getTransactions
HDDS-7175: Timed out tests missing from bundle
HDDS-7186: Add support for Rocksdb user info LOG configure
HDDS-6975: EC: Define the value of Maintenance Redundancy for EC containers
HDDS-7163: Ozone debug container CLI supports container export
HDDS-7176: Sort the container replica according to the replica index
HDDS-6312: Use KeyPrefixContainer table to accelerate the process of DELETE/UPDATE events
HDDS-6621: Include annotation processor module in regular project hierarchy
HDDS-7158: ldb cli command supports to scan container V3
HDDS-2954: Support admin groups
HDDS-7165: Remove AbstractContractUnbufferTest
HDDS-6493: Improve TestOmKeyInfoCodec error handling
HDDS-6967: EC: Handle maintenance replicas in ECUnderReplicationHandler
HDDS-7132: GetFileStatus returns NULL for some paths
HDDS-7172: Intermittent failure in Close pipeline smoketest
HDDS-7173: Fix TestReconOmMetadataManagerImpl#testUpdateOmDB
HDDS-7166: Memory leak in Recon when replacing DB for new checkpoint
HDDS-7161: Make Checksum.int2ByteString() zero-copy
HDDS-6243: sourceToTargetMap in ContainerBalancer doesn't support multiple entries with same source
HDDS-7162: Add Double in ConfigType
HDDS-7104: Remove WARNINGs while running ozone fs/CLI commands
HDDS-6456: Update RocksDB Version to 7.x
HDDS-7164: Skip installing awscli if already present
HDDS-7153: User defined override of ozone shell hardcoded values for log files
HDDS-7151: Avoid using GeneratedMessage in non-generated code
HDDS-7159: Use Java 8 for integration tests
HDDS-7157: Log the error msg explicitly in BlockDeletingService
HDDS-7159: Upgrade ubuntu version from 18.04 to 20.04
HDDS-7108: Make ReplicationConfig validation error message type-specific
HDDS-7150: Freon fail-at-end option caused not shutdown
HDDS-7154: Fixed code issues in org.apache.hadoop.HDDS.client reported by sonar
HDDS-5779: Create s3g bucket creation performance test
HDDS-7135: ofs file input stream should support StreamCapabilities interface
Multiple RPC clients per test run
HDDS-7108: Include supported EC data-parity combinations in error message
HDDS-7139: INVALID_TOKEN is handled as internal error in S3G
HDDS-7136: Memory leak due to ChunkInputStream.close() not releasing buffer
HDDS-7147: DirectoryDeletingService interval unit mismatch
HDDS-7113: Support overriding response header values
HDDS-7134: NPE when Ranger client throws RangerServiceException without Status
HDDS-6458: Add ReplicationConfig options for bucketArgs in MiniOzoneChaosCluster
HDDS-7076: Log container file path when container cannot be written
HDDS-7045: Election info is out of date in Recon
HDDS-7120: Prometheus displays only one volume_io_stats per datanode
HDDS-7074: DN EndpointStateMachineMBean to add getType()
HDDS-7127: Fix Transparent Data Encryption documentation
HDDS-7033: Include committed space in log for disk out of space
HDDS-7102: Exclude unnecessary RangerClient dependencies
HDDS-6726: Close RocksObject in Recon and tools
HDDS-6127: file checksum to support both CRC32 and CRC32C
HDDS-5504: Refactor namespace summaries (du, dist and counts) for upcoming legacy FS buckets support
HDDS-7116: Avoid leaking RocksObject from DBProfile
HDDS-7022: EC: Open EC container are not closed when SCM container was already closed
HDDS-6581: Introduce KEY_PATH_LOCK in OMKeyCreateRequest class
HDDS-7043: Disallow ellipsis in PR title
HDDS-7053: Add client-side pipelines distribution metrics
HDDS-7109: SCM should close LayoutVersionManager in stopping stage
HDDS-7047: Add Recon to ozone-topology sample compose environment
HDDS-6997: Add support for "echo" lookup to measure raw performance in OM
HDDS-7066: Include number of tests requested in Freon output
HDDS-7087: Manage RocksObjects to detect leaks
HDDS-7085: Update gRPC to 1.48.1 to address OOM bug
HDDS-7023: Add CLI for reseting the expired deleted block txn
HDDS-4970: Significant overhead when DataNode is over-subscribed
HDDS-7063: Increment createContainer metric when container is created while writing chunk
HDDS-6662: Provide basic hash code generator for OBSKeyPathLockStrategy
HDDS-7051: Fix offset Condition in ECKeyOutputStream
HDDS-7081: EC: ReplicationManager - UnderRep handler should handle duplicate indexes
HDDS-7077: EC: Fix block deletion not allowed due to missing pipelineID
HDDS-7044: Ignore pr_title_check for selective checks
HDDS-6951: Replace bucket.listKeys() with bucket.listStatus() in OmBucketReadWriteKeyOps
HDDS-5564: feature/PrefixFSO.md translation
HDDS-7073: Incomplete audit log for KeyBlockToDelete
HDDS-7069: EC: ReplicationManager - Track nodes already used when handing under replication
HDDS-7048: EC: Add debug logging with exception info when stripe write failed
HDDS-7068: EC: Prematurely re-throwed the exception in reconstruction cleanup loop
HDDS-6533: support balancing EC container
HDDS-7059: S3G should not call lookup twice for Get Object
HDDS-7050: Recon "/missing" endpoint return limit too high
HDDS-6899: EC: remove warnings and errors from console during online reconstruction of data
HDDS-7055: NPE in ec.reconstruction.TokenHelper
HDDS-7052: Topology Dynamic List document issue
HDDS-7056: EC: Ensure replica index is maintained when replicating a container
HDDS-7057: EC: ReplicationManager - Over replication handler should set repIndex on delete cmds
HDDS-6728: SCM UI not showing correct HA roles
HDDS-6956: Initialize ozoneLockProvider earlier in OzoneManager
HDDS-6980: support block composite-crc checksum
HDDS-7042: Rebuilding tenant cache omits empty tenants
HDDS-7046: Leader count missing in Recon
HDDS-7041: Avoid ConcurrentModificationException in RepeatedOmKeyInfo
HDDS-6767: Add network location in Recon's datanode page
HDDS-7021: [Multi-Tenant] Fix warning message when OM BG Sync is unable to get Ozone service policy version from Ranger
HDDS-7036: Enable RocksDB stats in monitoring sample config
HDDS-6871: Bump netty to 4.1.79
HDDS-7034: File handle leak on datanode in PutBlock
HDDS-7024: Add queue size metric for OM double buffer
HDDS-5251: Refactor ContainerTestHelper factory methods
HDDS-7032: RDBStore#getUpdatesSince should throw SequenceNumberNotFou…
HDDS-7015: Ensure excluded nodes have network location when used in placement policies
HDDS-6995: Update ranger-intg to v2.3.0
HDDS-6807: Remove tcnative.version property
HDDS-6976: 0GB data moved by container balancer after successful iteration
HDDS-7016: EC: Implement the Over replication Processor
HDDS-7009: Avoid calling getBucket in object APIs
HDDS-7020: EC: ReplicationManager - skip processing open containers
HDDS-6904: Cleanup customer related events in S3G logs
HDDS-6895: EC: ReplicationManager - Logic to process the under replicated queue and assign work to DNs
HDDS-7007: Missing spaces in error messages
HDDS-7012: Bump aws-java-sdk-s3 to 1.12.261
HDDS-7006: Incorrect usedNamespace caused by exceeding bytes quota
HDDS-6987: EC: Implement RECOVERING Container Scrubber
HDDS-7005: Recursive delete through bucket link deletes linked data
Add missing recon address to docker-config file
HDDS-6994: Update Contributing guide
HDDS-6998: Ignore keys with higher updateID in OpenKeyDeleteRequest
HDDS-6433: Refactor OMFailoverProxyProvider to provide a base for OM and GrpcOM FailoverProxyProviders
HDDS-6909: [Multi-Tenant] Use RangerClient for Ranger operations
HDDS-6940: EC: Skip the EC container for balancer
HDDS-6990: AuthorizerLockImpl.java#tryWriteLockInOMRequest: move sanity args check to after tryWriteLockThrowOnTimeout()
HDDS-6999: Correct volume parameter description for freon test tools
HDDS-6989: EC: Add Test for RECOVERING container cleanup when failure
HDDS-6981: Add metrics for BlockDeletingService on Datanode
HDDS-6889: EC: put key command with EC replication can use ReplicationConfig validator
HDDS-6761: [SCM HA finalization] Handle restarts, crashes, and leader changes
HDDS-6992: ListKeys : SubPaths with a mixture of files and dirs breaks sorted order
HDDS-6993: [Multi-Tenant] Set QuotaInBytes and QuotaInNamespace during Tenant Create
HDDS-6954: Migrate simple tests in ozone-recon to JUnit5
HDDS-6725: Close Rocks objects properly in StorageContainerManager
HDDS-6991: Throw java.lang.NullPointerException when running setOwner
HDDS-6901: Configure HDDS volume reserved as percentage of the volume space
HDDS-6978: EC: Cleanup RECOVERING container on DN restarts
HDDS-6982: EC: Attempt to cleanup the RECOVERING container when reconstruction failed at coordinator
HDDS-6968: Addendum: [Multi-Tenant] Fix USER_MISMATCH error even on correct user
HDDS-6794: EC: Analyze and add putBlock even on non writing node in the case of partial single stripe
HDDS-6900: Propagate TimeoutException for all SCM HA Ratis calls
HDDS-6938: handle NPE when removing prefixAcl
HDDS-6960: EC: Implement the Over-replication Handler
HDDS-6979: Remove unused plexus dependency declaration
HDDS-6957: EC: ReplicationManager - priortise under replicated containers
HDDS-6723: Close Rocks objects properly in OzoneManager
HDDS-6942: Ozone Buckets/Objects created via S3 should not allow group access
HDDS-6965: Increase timeout for basic check
HDDS-6969: Add link to compose directory in smoketest README
HDDS-6970: EC: Ensure DatanodeAdminMonitor can handle EC containers during decommission
HDDS-6977: EC: Remove references to ContainerReplicaPendingOps in TestECContainerReplicaCount
HDDS-6217: Cleanup XceiverClientGrpc TODOs, and document how the client works and should be used
HDDS-6773: Cleanup TestRDBTableStore - fix checkstyle
HDDS-6773: Cleanup TestRDBTableStore
HDDS-6676: KeyValueContainerData#getProtoBufMessage() should set block count
HDDS-6959: Fix move timeout in latest iteration metric
HDDS-6879: Bucket layout as null on default layout type
HDDS-6974: Container report processing in Recon is single threaded
HDDS-6973: Refactor ObjectEndpoint
HDDS-6894: EC: ReplicationManager - Add class to handle under-replication and form a command for a datanode
HDDS-6968: [Multi-Tenant] Fix USER_MISMATCH error even on correct user
HDDS-6928: ozone container balancer CLI went in hung state due to deadlock
HDDS-6948: MultiTenantAccessAuthorizerRangerPlugin#deletePolicyByName should log and exit if getAccessPolicyByName returns null
HDDS-6934: OM crashed with OzoneManagerDoubleBuffer
HDDS-6945: EC: EC Reconstruction Command count queues should be included in DN heartbeat
HDDS-6949: Audit bucket ACL changes in S3 Gateway
HDDS-6947: Bump gson version to 2.9.0
HDDS-6699: EC: ReplicationManager - collect under and over replicated containers
HDDS-6943: Reject property change requests on link buckets
HDDS-6944: EC: Handle reconstructECContainersCommand in heartbeat
HDDS-6937: Close Rocks objects properly in TrashOzoneFilesystem
HDDS-6906: Fix kerberos config for secure smoke tests
HDDS-6941: Setting Bucket Property can corrupt bucket layout
HDDS-6835: EC: Provide correct example for EC in ozone.server.default.replication
HDDS-6812: Fix INTERNAL_ERROR message on failed write
HDDS-5916: Datanodes stuck in leader election in Kubernetes
HDDS-6873: EC: Add block token support for ECReconstructionCoordinator
HDDS-6936: Mark TestContainerStateMachineFailures#testApplyTransactionFailure as flaky
HDDS-6891: Add CapacityVolumeChoosingPolicy
HDDS-6876: Migrate test with rules in HDDS-common to JUnit5
HDDS-6932: [Multi-Tenant] Move Ranger plugin version to a separate tag
HDDS-6868: Add S3Auth information to thread local
HDDS-6877: Keep replication port unchanged when restarting datanode in MiniOzoneCluster
HDDS-6907: OFS should create buckets with FILE_SYSTEM_OPTIMIZED layout
HDDS-6875: Migrate parameterized tests in HDDS-common to JUnit5
HDDS-6924: OBJECT_STORE isn't flat namespaced
HDDS-6899: [EC] Remove warnings and errors from console during online reconstruction of data
HDDS-6695: Enable SCM Ratis by default for new clusters only
HDDS-4123: Integrate OM Open Key Cleanup Service Into Existing Code
HDDS-6882: Correct exit code for invalid arguments passed to command-line tools
HDDS-6890: EC: Fix potential wrong replica read with over-replicated container
HDDS-6902: Duplicate mockito-core entries in pom.xml
HDDS-6752: Migrate tests with rules in HDDS-server-scm to JUnit5
HDDS-6806: EC: Implement the EC Reconstruction coordinator
HDDS-6829: Limit the no of inflight replication tasks in SCM
HDDS-6898: [SCM HA finalization] Modify acceptance test configuration to speed up test finalization
HDDS-6577: Configurations to reserve HDDS volume space
HDDS-6870: Clean up isTenantAdmin to use UGI
HDDS-6872: TestAuthorizationV4QueryParser should pass offline
HDDS-6840: Add MetaData volume information to the SCM and OM - UI
HDDS-6697: EC: ReplicationManager - create class to detect EC container health issues
HDDS-6841: EC: Validate the server default configuration on Ozone manager startup
HDDS-6830: EC: SCMContainerPlacementRackScatter may choose fewer nodes than required
HDDS-6260: EC: Standalone containers should not move to quasi-closed
HDDS-6799: Refactor OMLockMetrics code in OzoneManagerLock
HDDS-6878: Only export span if Ozone tracer init
HDDS-6874: Migrate simple tests in HDDS-common to JUnit5
HDDS-6869: Use UGI when checking Upgrade priv
HDDS-6837: Avoid duplicate headers in HTTP response
HDDS-6845: Add CI check for pull request title
HDDS-6846: HeadOp ignored for link bucket
HDDS-6839: Migrate tests in HDDS-client to JUnit5