Hadoop
HDP 2.5.0 provides the following Apache patches:
HADOOP-1540: Support file exclusion list in distcp.
HADOOP-3733: "s3x:" URLs break when Secret Key contains a slash, even if encoded.
HADOOP-5323: Trash documentation should describe its directory structure and configurations.
HADOOP-5470: RunJar.unJar() should write the last modified time found in the jar entry to the uncompressed file.
HADOOP-7139: Allow appending to existing SequenceFiles
HADOOP-7713: dfs -count -q should label output column.
HADOOP-7817: RawLocalFileSystem.append() should give FSDataOutputStream with accurate .getPos().
HADOOP-8151: Error handling in snappy decompressor throws invalid exceptions.
HADOOP-8818: Use equals instead == in MD5MD5CRC32FileChecksum and TFileDumper.
HADOOP-8934: Shell command ls should include sort options.
HADOOP-9121: InodeTree.java has redundant check for vName while throwing exception.
HADOOP-9242: Duplicate surefire plugin config in HADOOP-common.
HADOOP-9477: Add posixGroups support for LDAP groups mapping service.
HADOOP-10035: Cleanup TestFilterFileSystem.
HADOOP-1023: Add more logging in WASB page blob read function.
HADOOP-1026: Logging bug in WASB causing out of index exception.
HADOOP-10300: Allowed deferred sending of call responses.
HADOOP-10365: BufferedOutputStream in FileUtil#unpackEntries() should be closed in finally block.
HADOOP-10406: TestIPC.testIpcWithReaderQueuing may fail.
HADOOP-10597: RPC Server signals backoff to clients when all request queues are full.
HADOOP-10775: Shell operations to fail with meaningful errors on windows if winutils.exe not found.
HADOOP-10823: TestReloadingX509TrustManager is flaky.
HADOOP-10965: Print fully qualified path in CommandWithDestination error messages.
HADOOP-10971: Add -C flag to make `hadoop fs -ls` print filenames only.
HADOOP-11031: Design document for credential provider API.
HADOOP-11098: [JDK8] Max Non Heap Memory default changed between JDK7 and 8.
HADOOP-11212: NetUtils.wrapException to handle SocketException explicitly.
HADOOP-11218: Add TLSv1.1,TLSv1.2 to KMS, HttpFS, SSLFactory.
HADOOP-11252: RPC client does not time out by default.
HADOOP-11262: Enable YARN to use S3A.
HADOOP-11361: Fix a race condition in MetricsSourceAdapter.updateJmxCache.
HADOOP-11404: Clarify the "expected client Kerberos principal is null" authorization message.
HADOOP-11491: HarFs incorrectly declared as requiring an authority.
HADOOP-11581: Multithreaded correctness Warnings #org.apache.hadoop.fs.shell.Ls.
HADOOP-11613: Remove commons-httpclient dependency from HADOOP-azure.
HADOOP-11628: SPNEGO auth does not work with CNAMEs in JDK8.
HADOOP-11685: StorageException complaining "no lease ID" during HBase distributed log splitting.
HADOOP-11687: Ignore x-* and response headers when copying an Amazon S3 object.
HADOOP-11692: Improve authentication failure WARN message to avoid user confusion.
HADOOP-11713: ViewFileSystem should support snapshot methods.
HADOOP-11717: Add Redirecting WebSSO behavior with JWT Token in Hadoop Auth.
HADOOP-11772: RPC Invoker relies on static ClientCache which has synchronized(this) blocks.
HADOOP-11785: Reduce the number of listStatus operation in distcp buildListing.
HADOOP-11802: DomainSocketWatcher thread terminates sometimes after there is an I/O error during requestShortCircuitShm.
HADOOP-11812: Implement listLocatedStatus for ViewFileSystem to speed up split calculation.
HADOOP-11827: Speed-up distcp buildListing() using threadpool.
HADOOP-11859: PseudoAuthenticationHandler fails with httpcomponents v4.4.
HADOOP-11864: JWTRedirectAuthenticationHandler breaks java8 javadocs.
HADOOP-11876: Refactor code to make it more readable, minor maybePrintStats bug.
HADOOP-11901: BytesWritable fails to support 2G chunks due to integer overflow.
HADOOP-11918: Listing an empty s3a root directory throws FileNotFound.
HADOOP-11924: Tolerate JDK-8047340-related exceptions in Shell#isSetSidAvailable preventing class init.
HADOOP-11932: MetricsSinkAdapter may hang when being stopped.
HADOOP-11960: Enable Azure-Storage Client Side logging.
HADOOP-12001: Fixed LdapGroupsMapping to include configurable Posix UID and GID attributes during the search.
HADOOP-12006: Remove unimplemented option for from document.
HADOOP-12043: Display warning if defaultFs is not set when running fs commands.
HADOOP-12045: Enable LocalFileSystem#setTimes to change atime.
HADOOP-12049: Control http authentication cookie persistence via configuration.
HADOOP-12052: IPC client downgrades all exception types to IOE, breaks callers trying to use them.
HADOOP-12058: Fix dead links to DistCp and Hadoop Archives pages.
HADOOP-12059: S3Credentials should support use of CredentialProvider.
HADOOP-12061: Incorrect command in single cluster setup document.
HADOOP-12073: Azure FileSystem PageBlobInputStream does not return -1 on EOF.
HADOOP-12074: In Shell.java#runCommand() rethrow InterruptedException as InterruptedIOException.
HADOOP-12076: Incomplete cache mechanism in CredentialProvider API.
HADOOP-12089: StorageException complaining " no lease ID" when updating FolderLastModifiedTime in WASB.
HADOOP-12095: org.apache.hadoop.fs.shell.TestCount fails.
HADOOP-12100: ImmutableFsPermission should not override applyUmask since that method doesn't modify the FsPermission.
HADOOP-12103: Small refactoring of DelegationTokenAuthenticationFilter to allow code sharing.
HADOOP-12107: Long running apps may have a huge number of StatisticsData instances under FileSystem.
HADOOP-12119: Hadoop fs -expunge does not work for federated namespace.
HADOOP-12159: Move DistCpUtils#compareFs() to org.apache.hadoop.fs.FileUtil and fix for HA namespaces.
HADOOP-12161: Add getStoragePolicy API to the FileSystem interface.
HADOOP-12169: ListStatus on empty dir in S3A lists itself instead of returning an empty list.
HADOOP-12186: ActiveStandbyElector shouldn't call monitorLockNodeAsync multiple times.
HADOOP-12189: Improve CallQueueManager#swapQueue to make queue elements drop nearly impossible.
HADOOP-12191: Bzip2Factory is not thread safe.
HADOOP-12213: Interrupted exception can occur when Client#stop is called.
HADOOP-12230: HADOOP-project declares duplicate, conflicting curator dependencies.
HADOOP-12232: Upgrade Tomcat dependency to 6.0.44.
HADOOP-12239: StorageException complaining " no lease ID" when updating FolderLastModifiedTime in WASB.
HADOOP-12258: Need translate java.nio.file.NoSuchFileException to FileNotFoundException to avoid regression.
HADOOP-12259: Utility to Dynamic port allocation.
HADOOP-12269: Update aws-sdk dependency to 1.10.6.
HADOOP-12280: Skip unit tests based on maven profile rather than NativeCodeLoader.isNativeCodeLoaded.
HADOOP-12284: UserGroupInformation doAs can throw misleading exception.
HADOOP-12291: Add support for nested groups in LdapGroupsMapping.
HADOOP-12292: Make use of DeleteObjects optional.
HADOOP-12296: When setnetgrent returns 0 in Linux, exception should be thrown.
HADOOP-12304: Applications using FileContext fail with the default file system configured to be wasb/s3/etc.
HADOOP-12317: Applications fail on NM restart on some Linux distro because NM container recovery declares AM container as LOST.
HADOOP-12324: Better exception reporting in SaslPlainServer.
HADOOP-12325: RPC Metrics : Add the ability track and log slow RPCs.
HADOOP-12334: Change Mode Of Copy Operation of HBase WAL Archiving to bypass Azure Storage Throttling after retries.
HADOOP-12345: Pad hostname correctly in CredentialsSys.java.
HADOOP-12346: Increase some default timeouts / retries for S3a connector.
HADOOP-12348: MetricsSystemImpl creates MetricsSourceAdapter with wrong time unit parameter.
HADOOP-12350: WASB Logging: Improve WASB Logging around deletes, reads and writes.
HADOOP-12358: Add -safely flag to rm to prompt when deleting many files.
HADOOP-12359: hadoop fs -getmerge doc is wrong.
HADOOP-12366: Expose calculated paths (aw).
HADOOP-12374: Description of hdfs expunge command is confusing.
HADOOP-12406: Fixed AbstractMapWritable.readFields to use the thread's ClassLoader to load class instead of System ClassLoader.
HADOOP-12407: Test failing: hadoop.ipc.TestSaslRPC.
HADOOP-12413: AccessControlList should avoid calling getGroupNames in isUserInList with empty groups.
HADOOP-12415: Add io.netty dependency to HADOOP-nfs and to HADOOP-hdfs, needed to build Bigtop successfully, see BIGTOP-2049.
HADOOP-12416: Trash messages should be handled by Logger instead of being delivered on System.out.
HADOOP-12423: Handle failure of registering shutdownhook by ShutdownHookManager in static block
HADOOP-12426: Add Entry point for Kerberos health check.
HADOOP-12437: Allow SecurityUtil to lookup alternate hostnames.
HADOOP-12438: TestLocalFileSystem tests can fail on Windows after HDFS-8767 fix for handling pipe.
HADOOP-12440: TestRPC#testRPCServerShutdown did not produce the desired thread states before shutting down.
HADOOP-12441: Fixed kill-command behavior to work correctly across OSes by using bash shell built-in.
HADOOP-12444: Support lazy seek in S3AInputStream.
HADOOP-12463: Fix TestShell.testGetSignalKillCommand failure on windows.
HADOOP-12464: Interrupted client may try to fail-over and retry.
HADOOP-12465: Incorrect javadoc in WritableUtils.java.
HADOOP-12469: distcp should not ignore the ignoreFailures option.
HADOOP-12472: Make GenericTestUtils.assertExceptionContains robust.
HADOOP-12478: Shell.getWinUtilsPath() has been renamed Shell.getWinutilsPath().
HADOOP-12481: JWTRedirectAuthenticationHandler doesn't Retain Original Query String.
HADOOP-12482: Race condition in JMX cache update.
HADOOP-12484: Single File Rename Throws Incorrectly In Potential Race Condition Scenarios.
HADOOP-12508: Delete fails with exception when lease is held on blob.
HADOOP-12520: Use XInclude in HADOOP-azure test configuration to isolate Azure Storage account keys for service integration tests.
HADOOP-12526: There are duplicate dependency definitions in poms.
HADOOP-12533: Introduce FileNotFoundException in WASB for read and seek API.
HADOOP-12535: Run FileSystem contract tests with HADOOP-azure.
HADOOP-12537: S3A to support Amazon STS temporary credentials.
HADOOP-12540: TestAzureFileSystemInstrumentation#testClientErrorMetrics fails intermittently due to assumption that a lease error will be thrown.
HADOOP-12542: TestDNS fails on Windows after HADOOP-12437.
HADOOP-12545: Hadoop javadoc has broken links for AccessControlList, ImpersonationProvider, DefaultImpersonationProvider, and DistCp.
HADOOP-12548: Read s3a creds from a Credential Provider.
HADOOP-12551: Introduce FileNotFoundException for WASB FileSystem API.
HADOOP-12555: WASB to read credentials from a credential provider.
HADOOP-12559: KMS connection failures should trigger TGT renewal.
HADOOP-12564: Upgrade JUnit3 TestCase to JUnit 4 in org.apache.hadoop.io package.
HADOOP-12565: Replace DSA with RSA for SSH key type in SingleCluster.md.
HADOOP-12568: Update core-default.xml to describe posixGroups support.
HADOOP-12570: HDFS Secure Mode Documentation updates.
HADOOP-12573: TestRPC.testClientBackOff failing.
HADOOP-12577: Bump up commons-collections version to 3.2.2 to address a security flaw.
HADOOP-12588: Fix intermittent test failure of TestGangliaMetrics.
HADOOP-12589: Fix intermittent test failure of TestCopyPreserveFlag.
HADOOP-12590: TestCompressorDecompressor failing without stack traces.
HADOOP-12602: TestMetricsSystemImpl#testQSize occasionally fails.
HADOOP-12603: TestSymlinkLocalFSFileContext#testSetTimesSymlinkToDir occasionally fail.
HADOOP-12604: Exception may be swallowed in KMSClientProvider.
HADOOP-12605: Fix intermittent failure of TestIPC.testIpcWithReaderQueuingi.
HADOOP-12608: Fix exception message in WASB when connecting with anonymous credential.
HADOOP-12609: Fix intermittent failure of TestDecayRpcScheduler.
HADOOP-12613: TestFind.processArguments occasionally fails.
HADOOP-12622: Improve the loggings in RetryPolicies and RetryInvocationHandler.
HADOOP-12635: Adding Append API support for WASB.
HADOOP-12636: Prevent ServiceLoader failure init for unused FileSystems.
HADOOP-12653: Use SO_REUSEADDR to avoid getting "Address already in use" when using kerberos and attempting to bind to any port on the local IP address.
HADOOP-12659: Incorrect usage of config parameters in token manager of KMS.
HADOOP-12663: Remove Hard-Coded Values From FileSystem.java.
HADOOP-12668: Support excluding weak Ciphers in HttpServer2 through ssl- server.conf.
HADOOP-12672: RPC timeout should not override IPC ping interval.
HADOOP-12675: Fix description about retention period in usage of expunge command.
HADOOP-12678: Handle empty rename pending metadata file during atomic rename in redo path.
HADOOP-12682: Fix TestKMS#testKMSRestart* failure.
HADOOP-12688: Fix dead links in Compatibility.md.
HADOOP-12689: S3 filesystem operations stopped working correctly.
HADOOP-12691: Add CSRF Filter for REST APIs to Hadoop Common.
HADOOP-12691: Move files to correct location.
HADOOP-12696: Add tests for S3Filesystem Contract.
HADOOP-12699: TestKMS#testKMSProvider intermittently fails during 'test rollover draining'.
HADOOP-12700: Remove unused import in TestCompressorDecompressor.java.
HADOOP-12706: TestLocalFsFCStatistics#testStatisticsThreadLocalDataCleanUp times out occasionally.
HADOOP-12711: Remove dependency on commons-httpclient for ServletUtil.
HADOOP-12715: TestValueQueue#testgetAtMostPolicyALL fails intermittently.
HADOOP-12716: KerberosAuthenticator#doSpnegoSequence use incorrect class to determine isKeyTab in JDK8.
HADOOP-12717: NPE when trying to rename a directory in Windows Azure Storage FileSystem.
HADOOP-12723: S3A: Add ability to plug in any AWSCredentialsProvider.
HADOOP-12731: Remove useless boxing/unboxing code.
HADOOP-12736: TestTimedOutTestsListener#testThreadDumpAndDeadlocks sometimes times out.
HADOOP-12751: While using kerberos Hadoop incorrectly assumes names with '@' to be non-simple.
HADOOP-12752: Improve diagnostics/use of envvar/sysprop credential propagation.
HADOOP-12758: Extend CSRF Filter with UserAgent Checks.
HADOOP-12761: Incremental maven build is not really incremental.
HADOOP-12767: Update apache httpclient version to 4.5.2; httpcore to 4.4.4.
HADOOP-12772: NetworkTopologyWithNodeGroup.getNodeGroup() can loop infinitely for invalid 'loc' values.
HADOOP-12773: HBase classes fail to load with client/job classloader enabled.
HADOOP-12776: Remove getaclstatus call for non-acl commands in getfacl.
HADOOP-12780: During WASB atomic rename handle crash when one directory has been renamed but not file under it.
HADOOP-12782: Faster LDAP group name resolution with ActiveDirectory.
HADOOP-12786: "hadoop key" command usage is not documented.
HADOOP-12787: KMS SPNEGO sequence does not work with WEBHDFS.
HADOOP-12788: OpensslAesCtrCryptoCodec should log which random number generator is used.
HADOOP-12789: Log classpath of ApplicationClassLoader at INFO level.
HADOOP-12792: TestUserGroupInformation#testGetServerSideGroups fails in chroot.
HADOOP-12793: Write a new group mapping service guide.
HADOOP-12794: Support additional compression levels for GzipCodec.
HADOOP-12795: KMS does not log detailed stack trace for unexpected errors.
HADOOP-12800: Copy docker directory from 2.8 to 2.7/2.6 repos to enable pre-commit Jenkins runs.
HADOOP-12801: Suppress obsolete S3FileSystem tests.
HADOOP-12805: Annotate CanUnbuffer with @InterfaceAudience.Public
HADOOP-12807: S3AFileSystem should read AWS credentials from environment variables.
HADOOP-12810: FileSystem#listLocatedStatus causes unnecessary RPC calls
HADOOP-12817: Enable TLS v1.1 and 1.2.
HADOOP-12825: Log slow name resolutions.
HADOOP-12828: Print user when services are started.
HADOOP-12829: StatisticsDataReferenceCleaner swallows interrupt exceptions.
HADOOP-12831: LocalFS/FSOutputSummer NPEs in constructor if bytes per checksum set to 0.
HADOOP-12844: Recover when S3A fails on IOException in read().
HADOOP-12846: Credential Provider Recursive Dependencies.
HADOOP-12847: hadoop daemonlog should support https and SPNEGO for Kerberized cluster.
HADOOP-12849: TestSymlinkLocalFSFileSystem fails intermittently.
HADOOP-12851: S3AFileSystem Uptake of ProviderUtils.excludeIncompatibleCredentialProviders.
HADOOP-12853: Change WASB documentation regarding page blob support.
HADOOP-12870: Fix typo admininistration in CommandsManual.md.
HADOOP-12871: Fix dead link to NativeLibraries.html in CommandsManual.md.
HADOOP-12872: Fix formatting in ServiceLevelAuth.md.
HADOOP-12878: KMS SPNEGO sequence does not work with WEBHDFS.
HADOOP-12886: Exclude weak ciphers in SSLFactory through ssl-server.xml.
HADOOP-12891: S3AFileSystem should configure Multipart Copy threshold and chunk size.
HADOOP-12893: Update LICENSE.txt and NOTICE.txt.
HADOOP-12895: SSLFactory#createSSLSocketFactory exception message is wrong.
HADOOP-12901: Add warning log when KMSClientProvider cannot create a connection to the KMS server.
HADOOP-12902: JavaDocs for SignerSecretProvider are out-of-date in AuthenticationFilter.
HADOOP-12903: IPC Server should allow suppressing exception logging by type.
HADOOP-12903: IPC Server should allow suppressing exception logging by type, not log 'server too busy' messages.
HADOOP-12906: AuthenticatedURL should convert a 404/Not Found into an FileNotFoundException.
HADOOP-12909: Change ipc.Client to support asynchronous calls.
HADOOP-12916: Allow RPC scheduler/callqueue backoff using response times.
HADOOP-12926: lz4.c does not detect 64-bit mode properly.
HADOOP-12929: JWTRedirectAuthenticationHandler must accommodate null expiration time.
HADOOP-12943: Add -w -r options in dfs -test command.
HADOOP-12947: Update documentation Hadoop Groups Mapping to add static group mapping, negative cache.
HADOOP-12950: ShutdownHookManager should have a timeout for each of the Registered shutdown hook.
HADOOP-12957: Limit the number of outstanding async calls.
HADOOP-12958: PhantomReference for filesystem statistics can trigger OOM.
HADOOP-12962: KMS key names are incorrectly encoded when creating key.
HADOOP-12963: Allow using path style addressing for accessing the s3 endpoint.
HADOOP-12964: HTTP server vulnerable to clickjacking.
HADOOP-12973: Make DU pluggable.
HADOOP-12975: Add jitter to CachingGetSpaceUsed's thread.
HADOOP-12982: Document missing S3A and S3 properties.
HADOOP-12984: Add GenericTestUtils.getTestDir method and use it for temporary directory in tests.
HADOOP-12985: Support MetricsSource interface for DecayRpcScheduler Metrics.
HADOOP-12989: Some tests in org.apache.hadoop.fs.shell.find occasionally time out.
HADOOP-12993: Change ShutdownHookManger complete shutdown log from INFO to DEBUG.
HADOOP-12994: Specify PositionedReadable, add contract tests, fix problems.
HADOOP-13008: Add XFS Filter for UIs to Hadoop Common.
HADOOP-13011: Clearly Document the Password Details for Keystore-based Credential Providers.
HADOOP-13026: Should not wrap IOExceptions into a AuthenticationException in KerberosAuthenticator.
HADOOP-13028: Add low level counter metrics for S3A; use in read performance tests.
HADOOP-13030: Handle special characters in passwords in KMS startup script.
HADOOP-13039: Add documentation for configuration property ipc.maximum.data.length
HADOOP-13042: Restore lost leveldbjni LICENSE and NOTICE changes.
HADOOP-13047: S3a Forward seek in stream length to be configurable.
HADOOP-13052: ChecksumFileSystem mishandles crc file permissions.
HADOOP-13058: S3A FS fails during init against a read-only FS if multipart purge.
HADOOP-13065: Add a new interface for retrieving FS and FC Statistics.
HADOOP-13072: WindowsGetSpaceUsed constructor should be public.
HADOOP-13079: Add -q option to Ls to print ? instead of non-printable characters.
HADOOP-13084: Fix ASF License warnings.
HADOOP-13098: Dynamic LogLevel setting page should accept case-insensitive log level string.
HADOOP-13103: Group resolution from LDAP may fail on javax.naming.ServiceUnavailableException.
HADOOP-13105: Support timeouts in LDAP queries in LdapGroupsMapping.
HADOOP-13113: Enable parallel test execution for HADOOP-aws.
HADOOP-13116: Jets3tNativeS3FileSystemContractTest does not run.
HADOOP-13122: Customize User-Agent header sent in HTTP requests by S3A.
HADOOP-13130: s3a failures can surface as RTEs, not IOEs.
HADOOP-13131: Add tests to verify that S3A supports SSE-S3 encryption.
HADOOP-13138: Unable to append to a SequenceFile with Compression.NONE.
HADOOP-13140: FileSystem#initialize must not attempt to create StorageStatistics objects with null or empty schemes.
HADOOP-13145: In DistCp, prevent unnecessary getFileStatus call when not preserving metadata.
HADOOP-13146: Refactor RetryInvocationHandler.
HADOOP-13155: Implement TokenRenewer to renew and cancel delegation tokens in KMS.
HADOOP-13158: S3AFileSystem#toString might throw NullPointerException due to null cannedACL.
HADOOP-13159: Fix potential NPE in Metrics2 source for DecayRpcScheduler.
HADOOP-13162: Consider reducing number of getFileStatus calls in S3AFileSystem.mkdirs.
HADOOP-13163: Reuse pre-computed filestatus in Distcp-CopyMapper.
HADOOP-13168: Support Future.get with timeout in ipc async calls.
HADOOP-13171: Add StorageStatistics to S3A; instrument some more operations.
HADOOP-13179: GenericOptionsParser is not thread-safe because commons-cli OptionBuilder is not thread-safe.
HADOOP-13183: S3A proxy tests fail after httpclient/httpcore upgrade.
HADOOP-13192: org.apache.hadoop.util.LineReader cannot handle multibyte delimiters correctly.
HADOOP-13197: Add non-decayed call metrics for DecayRpcScheduler.
HADOOP-13203: S3a: Consider reducing the number of connection aborts by setting correct length in s3 request.
HADOOP-13213: Small Documentation bug with AuthenticatedURL in HADOOP-auth.
HADOOP-13237: s3a initialization against public bucket fails if caller lacks any credentials.
HADOOP-13241: Document s3a better.
HADOOP-13244: o.a.h.ipc.Server#Server should honor handlerCount when queueSizePerHandler is specified in constructor.
HADOOP-13249: RetryInvocationHandler need wrap InterruptedException in IOException when call Thread.sleep.
HADOOP-13251: Authenticate with Kerberos credentials when renewing KMS delegation token.
HADOOP-13255: KMSClientProvider should check and renew tgt when doing delegation token operations.
HADOOP-13263: Reload cached groups in background after expiry.
HADOOP-13270: BZip2CompressionInputStream finds the same compression marker twice in corner case, causing duplicate data blocks.
HADOOP-13280: FileSystemStorageStatistics#getLong(“readOps“) should return readOps + largeReadOps.
HADOOP-13283: Support reset operation for new global storage statistics and per FS storage stats.
HADOOP-13285: DecayRpcScheduler MXBean should only report decayed CallVolumeSummary.
HADOOP-13287: TestS3ACredentials#testInstantiateFromURL fails if AWS secret key contains +.
HADOOP-13288: Guard null stats key in FileSystemStorageStatistics.
HADOOP-13290: Appropriate use of generics in FairCallQueue.
HADOOP-13291: Probing stats in DFSOpsCountStatistics/S3AStorageStatistics should be correctly implemented.
HADOOP-13305: Define common statistics names across schemes.
HADOOP-13315: FileContext#umask is not initialized properly.
HADOOP-13316: Enforce Kerberos authentication for required ops in DelegationTokenAuthenticator.
HADOOP-13320: Fix arguments check in documentation for WordCount v2.0.
HADOOP-13350: Additional fix to LICENSE and NOTICE.
HADOOP-13351: TestDFSClientSocketSize buffer size tests are flaky.
HADOOP-13352: Make X-FRAME-OPTIONS configurable in HttpServer2.
HADOOP-13368: DFSOpsCountStatistics$OpType#fromSymbol and s3a.Statistic#fromSymbol should be O(1) operation.
HADOOP-13382: Remove unneeded commons-httpclient dependencies from POM files in Hadoop and sub-projects, while adding it in to the only place it is still needed, HADOOP-openstack.
HADOOP-13422: ZKDelegationTokenSecretManager JaasConfig does not work well with other ZK users in process.
HADOOP-13434: Add quoting to Shell class.
HADOOP-13467: Shell#getSignalKillCommand should use the bash built-in on Linux.
HDFS-1477: Support reconfiguring dfs.heartbeat.interval and dfs.namenode.heartbeat.recheck-interval without NN restart.
HDFS-2043: TestHFlush failing intermittently.
HDFS-2580: NameNode#main(.) can make use of GenericOptionsParser.
HDFS-27: HDFS CLI with --config set to default config complains log file not found error.
HDFS-3677: dfs.namenode.edits.dir.required is missing from HDFS- default.xml.
HDFS-4015: Safemode should count and report orphaned blocks.
HDFS-4185: Add a metric for number of active leases.
HDFS-4366: Block Replication Policy Implementation May Skip Higher-Priority Blocks for Lower-Priority Blocks.
HDFS-4396: Add START_MSG/SHUTDOWN_MSG for ZKFC.
HDFS-4937: ReplicationMonitor can infinite-loop in BlockPlacementPolicyDefault#chooseRandom.
HDFS-4946: Allow preferLocalNode in BlockPlacementPolicyDefault to be configurable.
HDFS-5640: Add snapshot methods to FileContext.
HDFS-6054: MiniQJMHACluster should not use static port to avoid binding failure in unit test.
HDFS-6101: TestReplaceDatanodeOnFailure fails occasionally.
HDFS-6407: Add sorting and pagination in the datanode tab of the NN Web UI.
HDFS-6481: DatanodeManager#getDatanodeStorageInfos() should check the length of storageIDs.
HDFS-6533: TestBPOfferService#testBasicFunctionalitytest fails intermittently.
HDFS-6581: Support for writing to single replica in RAM. Applied HDFS-6581 merge.15.patch to Champlain. This is a consolidated patch containing all feature changes and was applied to trunk.
HDFS-6666: Abort NameNode and DataNode startup if security is enabled but block access token is not enabled.
HDFS-6832: Fix the usage of 'hdfs namenode' command.
HDFS-6860: BlockStateChange logs are too noisy.
HDFS-6945: BlockManager should remove a block from excessReplicateMap and decrement ExcessBlocks metric when the block is removed.
HDFS-7163: WebHdfsFileSystem should retry reads according to the configured retry policy.
HDFS-7166: SbNN Web UI shows #Under replicated blocks and #pending deletion blocks.
HDFS-7314: When the DFSClient lease cannot be renewed, abort open-for- write files rather than the entire DFSClient.
HDFS-7390: Provide JMX metrics per storage type.
HDFS-7452: skip StandbyException log for getCorruptFiles()
HDFS-7483: Display information per tier on the Namenode UI.
HDFS-7597: DelegationTokenIdentifier should cache the TokenIdentifier to UGI mapping.
HDFS-7608: HDFS dfsclient newConnectedPeer has no write timeout.
HDFS-7609: Avoid retry cache collision when Standby NameNode loading edits.
HDFS-7645: Fix CHANGES.txt.
HDFS-7645: Rolling upgrade is restoring blocks from trash multiple times.
HDFS-7701: Support reporting per storage type quota and usage with hadoop/hdfs shell.
HDFS-7725: Incorrect "nodes in service" metrics caused all writes to fail.
HDFS-7833: DataNode reconfiguration does not recalculate valid volumes required, based on configured failed volumes tolerated.
HDFS-7858: Improve HA Namenode Failover detection on the client.
HDFS-7890: Improve information on Top users for metrics in RollingWindowsManager and lower log level.
HDFS-7923: The DataNodes should rate-limit their full block reports by asking the NN on heartbeat messages.
HDFS-7928: Scanning blocks from disk during rolling upgrade startup takes a lot of time if disks are busy.
HDFS-7933: fsck should also report decommissioning replicas.
HDFS-7978: Add LOG.isDebugEnabled() guard for some LOG.debug.
HDFS-7990: IBR delete ack should not be delayed.
HDFS-8008: Support client-side back off when the datanodes are congested.
HDFS-8009: Signal congestion on the DataNode.
HDFS-8046: Allow better control of getContentSummary.
HDFS-8055: NullPointerException when topology script is missing.
HDFS-8071: Redundant checkFileProgress() in PART II of getAdditionalBlock().
HDFS-8099: Change "DFSInputStream has been closed already" message to debug log level.
HDFS-8101: DFSClient use of non-constant DFSConfigKeys pulls in WebHDFS classes at runtime.
HDFS-8113: Add check for null BlockCollection pointers in BlockInfoContiguous structures.
HDFS-8131: Implement a space balanced block placement policy.
HDFS-8144: Split TestLazyPersistFiles into multiple tests.
HDFS-8152: Refactoring of lazy persist storage cases.
HDFS-8155: Support OAuth2 in WebHDFS.
HDFS-8180: AbstractFileSystem Implementation for WebHdfs.
HDFS-8205: CommandFormat#parse() should not parse option as value of option.
HDFS-8209: Support different number of datanode directories in MiniDFSCluster.
HDFS-8211: DataNode UUID is always null in the JMX counter.
HDFS-8219: setStoragePolicy with folder behavior is different after cluster restart.
HDFS-8229: LAZY_PERSIST file gets deleted after NameNode restart.
HDFS-8232: Missing datanode counters when using Metrics2 sink interface.
HDFS-8276: LazyPersistFileScrubber should be disabled if scrubber interval configured zero.
HDFS-8278: When computing max-size-to-move in Balancer, count only the storage with remaining >= default block size.
HDFS-8309: Skip unit test using DataNodeTestUtils#injectDataDirFailure() on Windows.
HDFS-8310: Fix TestCLI.testAll "help: help for find" on Windows.
HDFS-8311: DataStreamer.transfer() should timeout the socket InputStream.
HDFS-8337: Accessing httpfs via webhdfs doesn't work from a jar with Kerberos.
HDFS-8345: Storage policy APIs must be exposed via the FileSystem interface.
HDFS-8384: Allow NN to startup if there are files having a lease but are not under construction.
HDFS-8431: HDFS crypto class not found in Windows.
HDFS-8432: Introduce a minimum compatible layout version to allow downgrade in more rolling upgrade use cases.
HDFS-8435: Support CreateFlag in WebHDFS.
HDFS-8539: Hdfs doesn't have class 'debug' in windows.
HDFS-8542: WebHDFS getHomeDirectory behavior does not match specification.
HDFS-8546: Use try with resources in DataStorage and Storage.
HDFS-8548: Minicluster throws NPE on shutdown.
HDFS-8549: Abort the balancer if an upgrade is in progress.
HDFS-8554: TestDatanodeLayoutUpgrade fails on Windows.
HDFS-8578: On upgrade, Datanode should process all storage/data dirs in parallel.
HDFS-8581: ContentSummary on / skips further counts on yielding lock.
HDFS-8582: Support getting a list of reconfigurable config properties and do not generate spurious reconfig warnings.
HDFS-8593: Calculation of effective layout version mishandles comparison to current layout version in storage.
HDFS-8615: Correct HTTP method in WebHDFS document.
HDFS-8633: Fix setting of dfs.datanode.readahead.bytes in HDFS-default.xml to match DFSConfigKeys.
HDFS-8642: Make TestFileTruncate more reliable.
HDFS-8647: Abstract BlockManager's rack policy into BlockPlacementPolicy.
HDFS-8656: Preserve compatibility of ClientProtocol#rollingUpgrade after finalization.
HDFS-8659: Block scanner INFO message is spamming logs.
HDFS-8676: Delayed rolling upgrade finalization can cause heartbeat expiration.
HDFS-8696: Make the lower and higher watermark in the DN Netty server configurable.
HDFS-8709: Clarify automatic sync in FSEditLog#logEdit.
HDFS-8722: Optimize datanode writes for small writes and flushes.
HDFS-8729: Fix TestFileTruncate#testTruncateWithDataNodesRestartImmediately which occasionally failed.
HDFS-8767: RawLocalFileSystem.listStatus() returns null for UNIX pipefile.
HDFS-8772: Fix TestStandbyIsHot#testDatanodeRestarts which occasionally fails.
HDFS-8778: TestBlockReportRateLimiting#testLeaseExpiration can deadlock.
HDFS-8785: TestDistributedFileSystem is failing in trunk.
HDFS-8797: WebHdfsFileSystem creates too many connections for pread.
HDFS-8806: Inconsistent metrics: number of missing blocks with replication factor 1 not properly cleared.
HDFS-8809: HDFS fsck reports under construction blocks as CORRUPT.
HDFS-8815: DFS getStoragePolicy implementation using single RPC call.
HDFS-8816: Improve visualization for the Datanode tab in the NN UI.
HDFS-8818: Changes the global moveExecutor to per datanode executors and changes MAX_SIZE_TO_MOVE to be configurable.
HDFS-8824: Do not use small blocks for balancing the cluster.
HDFS-8826: In Balancer, add an option to specify the source node list so that balancer only selects blocks to move from those nodes.
HDFS-8828: Utilize Snapshot diff report to build diff copy list in distcp.
HDFS-8829: Make SO_RCVBUF and SO_SNDBUF size configurable for DataTransferProtocol sockets and allow configuring auto-tuning.
HDFS-8831: Trash Support for deletion in HDFS encryption zone.
HDFS-8844: TestHDFSCLI does not cleanup the test directory.
HDFS-8845: DiskChecker should not traverse the entire tree.
HDFS-8846: Add a unit test for INotify functionality across a layout version upgrade.
HDFS-8850: VolumeScanner thread exits with exception if there is no block pool to be scanned but there are suspicious blocks.
HDFS-8852: HDFS architecture documentation of version 2.x is outdated about append write support.
HDFS-8855: Webhdfs client leaks active NameNode connections.
HDFS-8856: Make LeaseManager#countPath O(1).
HDFS-8863: The remaining space check in BlockPlacementPolicyDefault is flawed.
HDFS-8867: Enable optimized block reports.
HDFS-8879: Quota by storage type usage incorrectly initialized upon namenode restart.
HDFS-8880: NameNode metrics logging.
HDFS-8883: NameNode Metrics : Add FSNameSystem lock Queue Length.
HDFS-8885: ByteRangeInputStream used in webhdfs does not override available().
HDFS-8887: Expose storage type and storage ID in BlockLocation.
HDFS-8891: HDFS concat should keep srcs order.
HDFS-8896: DataNode object isn't GCed when shutdown, because it has GC root in ShutdownHookManager.
HDFS-8911: NameNode Metric : Add Editlog counters as a JMX metric.
HDFS-8914: Document HA support in the HDFS HdfsDesign.md.
HDFS-8923: Add -source flag to balancer usage message.
HDFS-8930: Block report lease may leak if the 2nd full block report comes when NN is still in safemode.
HDFS-8932: NPE thrown in NameNode when try to get TotalSyncCount metric before editLogStream initialization.
HDFS-8939: Test(S)WebHdfsFileContextMainOperations failing.
HDFS-8950: NameNode refresh doesn't remove DataNodes that are no longer in the allowed list.
HDFS-8964: When validating the edit log, do not read at or beyond the file offset that is being written.
HDFS-8965: Harden edit log reading code against out of memory errors.
HDFS-8969: Clean up findbugs warnings for HDFS-8823 and HDFS-8932.
HDFS-8983: NameNode support for protected directories.
HDFS-8995: Flaw in registration bookkeeping can make DN die on reconnect.
HDFS-9008: Balancer#Parameters class could use a builder pattern.
HDFS-9009: Send metrics logs to NullAppender by default.
HDFS-9019: Adding informative message to sticky bit permission denied exception.
HDFS-9033: dfsadmin -metasave prints "NaN" for cache used%.
HDFS-9042: Update document for the Storage policy name.
HDFS-9043: Doc update for commands in HDFS Federation.
HDFS-9047: Retire libwebhdfs.
HDFS-9048: DistCp documentation is out-of-dated.
HDFS-9063: Correctly handle snapshot path for getContentSummary.
HDFS-9072: Fix random failures in TestJMXGet.
HDFS-9082: Change the log level in WebHdfsFileSystem.initialize() from INFO to DEBUG.
HDFS-9083: Replication violates block placement policy.
HDFS-9094: Add command line option to ask NameNode reload configuration.
HDFS-9106: Transfer failure during pipeline recovery causes permanent write failures.
HDFS-9107: Prevent NNs unrecoverable death spiral after full GC.
HDFS-9112: Improve error message for Haadmin when multiple name service IDs are configured.
HDFS-9128: TestWebHdfsFileContextMainOperations and TestSWebHdfsFileContextMainOperations fail due to invalid HDFS path on Windows.
HDFS-9130: Use GenericTestUtils#setLogLevel to the logging level.
HDFS-9142: Separating Configuration object for namenode(s) in MiniDFSCluster.
HDFS-9175: Change scope of 'AccessTokenProvider.getAccessToken()' and 'CredentialBasedAccessTokenProvider.getCredential()' abstract methods to public.
HDFS-9178: Slow datanode I/O can cause a wrong node to be marked bad.
HDFS-9184: Logging HDFS operation's caller context into audit logs.
HDFS-9198: Coalesce IBR processing in the NN.
HDFS-9205: Do not schedule corrupt blocks for replication.
HDFS-9210: Fix some misuse of %n in VolumeScanner#printStats.
HDFS-9214: Support reconfiguring dfs.datanode.balance.max.concurrent.moves without DN restart.
HDFS-9220: Reading small file (< 512 bytes) that is open for append fails due to incorrect checksum.
HDFS-9221: HdfsServerConstants#ReplicaState#getState should avoid calling values() since it creates a temporary array.
HDFS-9226: Unit Tests failing due to mockito version issue.
HDFS-9239: DataNode Lifeline Protocol: an alternative protocol for reporting DataNode liveness.
HDFS-9259: Make SO_SNDBUF size configurable at DFSClient side for hdfs write scenario.
HDFS-9273: ACLs on root directory may be lost after NN restart.
HDFS-9289: Make DataStreamer#block thread safe and verify genStamp in commitBlock.
HDFS-9290: DFSClient#callAppend() is not backward compatible for slightly older NameNodes.
HDFS-9294: DFSClient deadlock when close file and failed to renew lease.
HDFS-9305: Delayed heartbeat processing causes storm of subsequent heartbeats.
HDFS-9311: Support optional offload of NameNode HA service health checks to a separate RPC server.
HDFS-9313: Possible NullPointerException in BlockManager if no excess replica can be chosen.
HDFS-9314: Improve BlockPlacementPolicyDefault's picking of excess replicas.
HDFS-9317: Document fsck -blockId and -storagepolicy options.
HDFS-9343: Empty caller context considered invalid.
HDFS-9347: Invariant assumption in TestQuorumJournalManager.shutdown() is wrong.
HDFS-9349: Support reconfiguring fs.protected.directories without NN restart.
HDFS-9354: Fix TestBalancer#testBalancerWithZeroThreadsForMove on Windows.
HDFS-9358: TestNodeCount#testNodeCount timed out.
HDFS-9362: TestAuditLogger#testAuditLoggerWithCallContext assumes Unix line endings, fails on Windows.
HDFS-9364: Unnecessary DNS resolution attempts when creating NameNodeProxies.
HDFS-9383: TestByteArrayManager#testByteArrayManager fails.
HDFS-9384: TestWebHdfsContentLength intermittently hangs and fails due to TCP conversation mismatch between client and server.
HDFS-9395: Make HDFS audit logging consistent.
HDFS-9397: Fix typo for readChecksum() LOG.warn in BlockSender.java.
HDFS-9402: Switch DataNode.LOG to use slf4j.
HDFS-9406: FSImage may get corrupted after deleting snapshot.
HDFS-9412: getBlocks occupies FSLock and takes too long to complete.
HDFS-9413: getContentSummary() on standby should throw StandbyException.
HDFS-9414: Refactor reconfiguration of ClientDatanodeProtocol for reusability.
HDFS-9415: Document dfs.cluster.administrators and dfs.permissions.superusergroup.
HDFS-9426: Rollingupgrade finalization is not backward compatible.
HDFS-9428: Fix intermittent failure of TestDNFencing.testQueueingWithAppend.
HDFS-9431: DistributedFileSystem#concat fails if the target path is relative.
HDFS-9434: Recommission a datanode with 500k blocks may pause NN for 30 seconds for printing info log messages.
HDFS-9445: Datanode may deadlock while handling a bad volume.
HDFS-9466: TestShortCircuitCache#testDataXceiverCleansUpSlotsOnFailure is flaky.
HDFS-9470: Encryption zone on root not loaded from fsimage after NN restart.
HDFS-9476: TestDFSUpgradeFromImage#testUpgradeFromRel1BBWImage occasionally fail.
HDFS-9478: Reason for failing ipc.FairCallQueue construction should be thrown.
HDFS-9493: Test o.a.h.hdfs.server.namenode.TestMetaSave fails in trunk.
HDFS-9505: HDFS Architecture documentation needs to be refreshed.
HDFS-9516: Truncate file fails with data dirs on multiple disks.
HDFS-9517: Fix missing @Test annotation on TestDistCpUtils.testUnpackAttributes.
HDFS-9521: TransferFsImage.receiveFile should account and log separate times for image download and fsync to disk.
HDFS-9530: ReservedSpace is not cleared for abandoned Blocks.
HDFS-9533: seen_txid in the shared edits directory is modified during bootstrapping.
HDFS-9534: Add CLI command to clear storage policy from a path.
HDFS-9549: TestCacheDirectives#testExceedsCapacity is flaky.
HDFS-9555: LazyPersistFileScrubber should still sleep if there are errors in the clear progress.
HDFS-9557: Reduce object allocation in PB conversion.
HDFS-9566: Remove expensive 'BlocksMap#getStorages(Block b, final DatanodeStorage.State state)' method
HDFS-9569: Log the name of the fsimage being loaded for better supportability.
HDFS-9572: Prevent DataNode log spam if a client connects on the data transfer port but sends no data.
HDFS-9574: Reduce client failures during datanode restart.
HDFS-9584: NPE in distcp when ssl configuration file does not exist in class path.
HDFS-9600: Do not check replication if the block is under construction.
HDFS-9605: Add links to failed volumes to explorer.html in HDFS Web UI.
HDFS-9608: Disk IO imbalance in HDFS with heterogeneous storages.
HDFS-9612: DistCp worker threads are not terminated after jobs are done.
HDFS-9619: SimulatedFSDataset sometimes can not find blockpool for the correct namenode.
HDFS-9623: Update example configuration of block state change log in log4j.properties.
HDFS-9624: DataNode start slowly due to the initial DU command operations.
HDFS-9625: Set replication for empty file failed when set storage policy.
HDFS-9626: TestBlockReplacement#testBlockReplacement fails occasionally.
HDFS-9629: Update the footer of Web UI to show year 2016.
HDFS-9630: DistCp minor refactoring and clean up.
HDFS-9634: WebHDFS client side exceptions don't provide enough details.
HDFS-9638: Improve DistCp Help and documentation.
HDFS-9644: Update encryption documentation to reflect nested EZs.
HDFS-9655: NN should start JVM pause monitor before loading fsimage.
HDFS-9661: Deadlock in DN.FsDatasetImpl between moveBlockAcrossStorage and createRbw.
HDFS-9669: TcpPeerServer should respect ipc.server.listen.queue.size.
HDFS-9670: DistCp throws NPE when source is root.
HDFS-9672: o.a.h.hdfs.TestLeaseRecovery2 fails intermittently.
HDFS-9688: Test the effect of nested encryption zones in HDFS downgrade.
HDFS-9706: Log more details in debug logs in BlockReceiver's constructor.
HDFS-9710: DN can be configured to send block receipt IBRs in batches.
HDFS-9711: Integrate CSRF prevention filter in WebHDFS.
HDFS-9713: DataXceiver#copyBlock should return if block is pinned.
HDFS-9715: Check storage ID uniqueness on datanode startup
HDFS-9721: Allow Delimited PB OIV tool to run upon fsimage that contains INodeReference.
HDFS-9724: Degraded performance in WebHDFS listing as it does not reuse ObjectMapper.
HDFS-9726: Refactor IBR code to a new class.
HDFS-9730: Storage ID update does not happen when there is a layout change.
HDFS-9739: DatanodeStorage.isValidStorageId() is broken.
HDFS-9740: Use a reasonable limit in DFSTestUtil.waitForMetric().
HDFS-9743: Fix TestLazyPersistFiles#testFallbackToDiskFull.
HDFS-9748: Avoid duplication in pendingReplications when addExpectedReplicasToPending is called twice.
HDFS-9752: Permanent write failures may happen to slow writers during datanode rolling upgrades.
HDFS-9760: WebHDFS AuthFilter cannot be configured with custom AltKerberos auth handler.
HDFS-9764: DistCp doesn't print value for several arguments including -numListstatusThreads.
HDFS-9765: TestBlockScanner#testVolumeIteratorWithCaching fails intermittently.
HDFS-9766: TestDataNodeMetrics#testDataNodeTimeSpend fails intermittently.
HDFS-9768: Reuse ObjectMapper instance in HDFS to improve the performance.
HDFS-9772: TestBlockReplacement#testThrottler doesn't work as expected.
HDFS-9777: Fix typos in DFSAdmin command line and documentation.
HDFS-9779: TestReplicationPolicyWithNodeGroup NODE variable picks wrong rack value.
HDFS-9784: Example usage is not correct in Transparent Encryption document.
HDFS-9790: HDFS Balancer should exit with a proper message if upgrade is not finalized.
HDFS-9797: Log Standby exceptions thrown by RequestHedgingProxyProvider at DEBUG Level.
HDFS-9799: Reimplement getCurrentTrashDir to remove incompatibility.
HDFS-9801: ReconfigurableBase should update the cached configuration.
HDFS-9812: Streamer threads leak if failure happens when closing DFSOutputStream.
HDFS-9839: Reduce verbosity of processReport logging.
HDFS-9842: dfs.datanode.balance.bandwidthPerSec should accept friendly size units.
HDFS-9844: Correct path creation in getTrashRoot to handle root dir.
HDFS-9851: NameNode throws NPE when setPermission is called on a path that does not exist.
HDFS-9854: Log cipher suite negotiation more verbosely.
HDFS-9864: Correct reference for RENEWDELEGATIONTOKEN and CANCELDELEGATIONTOKEN in webhdfs doc.
HDFS-9865: TestBlockReplacement fails intermittently in trunk.
HDFS-9871: "Bytes Being Moved" -ve(-1 B) when cluster was already balanced.
HDFS-9874: Long living DataXceiver threads cause volume shutdown to block.
HDFS-9880: TestDatanodeRegistration fails occasionally.
HDFS-9881: DistributedFileSystem#getTrashRoot returns incorrect path for encryption zones.
HDFS-9882: Add heartbeatsTotal in Datanode metrics.
HDFS-9902: Support different values of dfs.datanode.du.reserved per storage type.
HDFS-9904: testCheckpointCancellationDuringUpload occasionally fails.
HDFS-9905: WebHdfsFileSystem#runWithRetry should display original stack trace on error.
HDFS-9906: Remove spammy log spew when a datanode is restarted.
HDFS-9917: IBR accumulate more objects when SNN was down for sometime.
HDFS-9926: MiniDFSCluster leaks dependency Mockito via DataNodeTestUtils.
HDFS-9941: Do not log StandbyException on NN, other minor logging fixes.
HDFS-9949: Add a test case to ensure that the DataNode does not regenerate its UUID when a storage directory is cleared.
HDFS-9958: BlockManager#createLocatedBlocks can throw NPE for corruptBlocks on failed storages.
HDFS-10178: Permanent write failures can happen if pipeline recoveries occur for the first packet.
HDFS-10182: Hedged read might overwrite user's buf.
HDFS-10186: DirectoryScanner: Improve logs by adding full path of both actual and expected block directories.
HDFS-10189: PacketResponder#toString should include the downstreams for PacketResponderType.HAS_DOWNSTREAM_IN_PIPELINE.
HDFS-10199: Unit tests TestCopyFiles, TestDistCh, TestLogalyzer under org.apache.hadoop.tools are failing.
HDFS-10207: Support enable Hadoop IPC backoff without namenode restart.
HDFS-10209: Support enable caller context in HDFS namenode audit log without restart namenode.
HDFS-10216: Distcp -diff throws exception when handling relative path.
HDFS-10217: Show 'blockScheduled' tooltip in datanodes table.
HDFS-10223: peerFromSocketAndKey performs SASL exchange before setting connection timeouts.
HDFS-10224: Implement asynchronous rename for DistributedFileSystem.
HDFS-10228: TestHDFSCLI fails.
HDFS-10235: Last contact for Live Nodes should be relative time.
HDFS-10239: Fsshell mv fails if port usage doesn't match in src and destination paths.
HDFS-10245: Fix the findbugs warnings.
HDFS-10253: Fix TestRefreshCallQueue failure.
HDFS-10261: TestBookKeeperHACheckpoints doesn't handle ephemeral HTTP ports.
HDFS-10264: Logging improvements in FSImageFormatProtobuf.Saver.
HDFS-10270: TestJMXGet:testNameNode() fails.
HDFS-10271: Extra bytes are getting released from reservedSpace for append.
HDFS-10275: TestDataNodeMetrics failing intermittently due to TotalWriteTime counted incorrectly.
HDFS-10277: PositionedReadable test testReadFullyZeroByteFile failing in HDFS.
HDFS-10281: TestPendingCorruptDnMessages fails intermittently.
HDFS-10291: TestShortCircuitLocalRead failing.
HDFS-10309: Balancer doesn't honor dfs.blocksize value defined with suffix k(kilo), m(mega), g(giga).
HDFS-10312: Large block reports may fail to decode at NameNode due to 64 MB protobuf maximum length restriction.
HDFS-10313: Distcp need to enforce the order of snapshot names passed to -diff.
HDFS-10319: Balancer should not try to pair storages with different types.
HDFS-10324: Trash directory in an encryption zone should be pre-created with correct permissions.
HDFS-10329: Bad initialisation of StringBuffer in RequestHedgingProxyProvider.
HDFS-10335: Mover$Processor#chooseTarget() always chooses the first matching target storage group.
HDFS-10341: Add a metric to expose the timeout number of pending replication blocks.
HDFS-10344: DistributedFileSystem#getTrashRoots should skip encryption zone that does not have .Trash.
HDFS-10346: Implement asynchronous setPermission/setOwner for DistributedFileSystem.
HDFS-10347: Namenode report bad block method doesn't log the bad block or datanode.
HDFS-10360: DataNode may format directory and lose blocks if current/VERSION is missing.
HDFS-10367: TestDFSShell.testMoveWithTargetPortEmpty fails with Address bind exception.
HDFS-10372: Fix for failing TestFsDatasetImpl#testCleanShutdownOfVolume.
HDFS-10377: CacheReplicationMonitor shutdown log message should use INFO level.
HDFS-10390: Implement asynchronous setAcl/getAclStatus for DistributedFileSystem.
HDFS-10396: Using -diff option with DistCp may get "Comparison method violates its general contract" exception.
HDFS-10397: Distcp should ignore -delete option if -diff option is provided instead of exiting.
HDFS-10410: RedundantEditLogInputStream.LOG is set to wrong class.
HDFS-10415: TestDistributedFileSystem#MyDistributedFileSystem attempts to set up statistics before initialize() is called.
HDFS-10417: Improve error message from checkBlockLocalPathAccess.
HDFS-10424: DatanodeLifelineProtocol not able to use under security cluster.
HDFS-10431: Refactor and speedup TestAsyncDFSRename.
HDFS-10437: ReconfigurationProtocol not covered by HDFSPolicyProvider.
HDFS-10438: When NameNode HA is configured to use the lifeline RPC server, it should log the address of that server.
HDFS-10440: Improve DataNode web UI.
HDFS-10448: CacheManager#addInternal tracks bytesNeeded incorrectly when dealing with replication factors other than 1.
HDFS-10458: getFileEncryptionInfo should return quickly for non-encrypted cluster.
HDFS-10468: HDFS read ends up ignoring an interrupt.
HDFS-10469: Add number of active xceivers to datanode metrics.
HDFS-10471: DFSAdmin#SetQuotaCommand's help msg is not correct.
HDFS-10474: hftp copy fails when file name with Chinese+special char.
HDFS-10481: HTTPFS server should correctly impersonate as end user to open file.
HDFS-10485: Fix findbugs warning in FSEditLog.java.
HDFS-10488: Update WebHDFS documentation regarding CREATE and MKDIR default permissions.
HDFS-10493: Add links to datanode web UI in namenode datanodes page.
HDFS-10508: DFSInputStream should set thread's interrupt status after catching InterruptException from sleep.
HDFS-10525: Fix NPE in CacheReplicationMonitor#rescanCachedBlockMap.
HDFS-10556: DistCpOptions should be validated automatically.
HDFS-10579: HDFS web interfaces lack configs for X-FRAME-OPTIONS protection.
HDFS-10589: Javadoc for HAState#HAState and HAState#setStateInternal contains non-existent params.
HDFS-10617: PendingReconstructionBlocks.size() should be synchronized.
HDFS-10623: Remove unused import of httpclient.HttpConnection from TestWebHdfsTokens.
HDFS-10642: TestLazyPersistReplicaRecovery#testDnRestartWithSavedReplicas fails intermittently.
HDFS-10643: HDFS namenode should always use service user (hdfs) to generateEncryptedKey.
HDFS-10660: Expose storage policy APIs via HDFSAdmin interface.
HDFS-10688: BPServiceActor may run into a tight loop for sending block report when hitting IOException.
MAPREDUCE-5485: Allow repeating job commit by extending OutputCommitter API.
MAPREDUCE-6197: Cache MapOutputLocations in ShuffleHandler.
MAPREDUCE-6273: HistoryFileManager should check whether summaryFile exists to avoid FileNotFoundException causing HistoryFileInfo into MOVE_FAILED state.
MAPREDUCE-6302: Backport preempt reducers after a configurable timeout irrespective of headroom.
MAPREDUCE-6436: JobHistory cache issue.
MAPREDUCE-6460: TestRMContainerAllocator.testAttemptNotFoundCausesRMCommunicatorException fails.
MAPREDUCE-6492: AsyncDispatcher exit with NPE on TaskAttemptImpl#sendJHStartEventForAssignedFailTask.
MAPREDUCE-6514: Job hangs as ask is not updated after ramping down of all reducers.
MAPREDUCE-6543: Migrate MR Client test cases part 2.
MAPREDUCE-6549: multibyte delimiters with LineRecordReader cause duplicate records.
MAPREDUCE-6566: Add retry support to mapreduce CLI tool.
MAPREDUCE-6577: MR AM unable to load native library without MR_AM_ADMIN_USER_ENV set.
MAPREDUCE-6579: TestNetworkedJob.testNetworkedJob failed due to YARN-3946.
MAPREDUCE-6593: TestJobHistoryEventHandler.testTimelineEventHandling fails because of NPE.
MAPREDUCE-6618: YarnClientProtocolProvider leaking the YarnClient thread.
MAPREDUCE-6621: Memory Leak in JobClient#submitJobInternal().
MAPREDUCE-6635: Unsafe long to int conversion in UncompressedSplitLineReader and IndexOutOfBoundsException.
MAPREDUCE-6639: Process hangs in LocatedFileStatusFetcher if FileSystem.get throws.
MAPREDUCE-6672: TestTeraSort fails on Windows.
MAPREDUCE-6689: MapReduce job can infinitely increase number of reducer resource requests.
YARN-998: Keep NM resource updated through dynamic resource config for RM/NM restart.
YARN-1376: NM need to notify the log aggregation status to RM through heartbeat.
YARN-1402: Update related Web UI and CLI with exposing client API to check log aggregation status.
YARN-1462: AHS API and other AHS changes to handle tags for completed MR jobs.
YARN-1556: NPE getting application report with a null appId.
YARN-1880: Cleanup TestApplicationClientProtocolOnHA.
YARN-2019: EAR-1682. Retrospect on decision of making RM crashed if any exception throw in ZKRMStateStore.
YARN-2046: Out of band heartbeats are sent only on container kill and possibly too early.
YARN-2194: Fix bug causing CGroups functionality to fail on RHEL7.
YARN-2392: Add more diags about app retry limits on AM failures.
YARN-2495: Allow admin specify labels from each NM (Distributed configuration for node label).
YARN-2498: Respect labels in preemption policy of capacity scheduler for inter-queue preemption.
YARN-2513: Host framework UIs in YARN for use with the ATS Note: Patch is in progress.
YARN-2571: RM to support YARN registry.
YARN-2619: Added NodeManager support for disk IO isolation through cgroups.
YARN-2666: TestFairScheduler.testContinuousScheduling fails intermittently.
YARN-2696: Queue sorting in CapacityScheduler should consider node label.
YARN-2740: Fix NodeLabelsManager to properly handle node label modifications when distributed node label configuration enabled.
YARN-2821: Fixed a problem that DistributedShell AM may hang if restarted.
YARN-2868: FairScheduler: Metric for latency to allocate first container for an application.
YARN-2901: Add errors and warning metrics page to RM, NM web UI.
YARN-3021: YARN's delegation-token handling disallows certain trust setups to operate properly over DistCp.
YARN-3026: Move application-specific container allocation logic from LeafQueue to FiCaSchedulerApp.
YARN-3102: Decommissioned Nodes not listed in Web UI.
YARN-3110: Few issues in ApplicationHistory web UI.
YARN-3136: Fixed a synchronization problem of AbstractYarnScheduler#getTransferredContainers.
YARN-3215: Respect labels in CapacityScheduler when computing headroom.
YARN-3216: Max-AM-Resource-Percentage should respect node labels.
YARN-3225: New parameter of CLI for decommissioning node gracefully in RMAdmin CLI.
YARN-3243: CapacityScheduler should pass headroom from parent to children to make sure ParentQueue obey its capacity limits.
YARN-3248: Display count of nodes blacklisted by apps in the web UI.
YARN-3266: RMContext#inactiveNodes should have NodeId as map key.
YARN-3269: Yarn.nodemanager.remote-app-log-dir could not be configured to fully qualified path.
YARN-3293: Track and display capacity scheduler health metrics in web UI.
YARN-3294: Allow dumping of Capacity Scheduler debug logs via web UI for a fixed time period.
YARN-3305: Normalize AM resource request on app submission.
YARN-3318: Create Initial OrderingPolicy Framework and FifoOrderingPolicy.
YARN-3319: Implement a FairOrderingPolicy.
YARN-3326: Support RESTful API for getLabelsToNodes.
YARN-3343: Increased TestCapacitySchedulerNodeLabelUpdate#testNodeUpdate timeout.
YARN-3345: Add non-exclusive node label API.
YARN-3347: Improve YARN log command to get AMContainer logs as well as running containers logs.
YARN-3348: Add a 'yarn top' tool to help understand cluster usage.
YARN-3354: Add node label expression in ContainerTokenIdentifier to support RM recovery.
YARN-3356: Capacity Scheduler FiCaSchedulerApp should use ResourceUsage to track used-resources-by-label.
YARN-3357: Move TestFifoScheduler to FIFO package.
YARN-3360: Add JMX metrics to TimelineDataManager.
YARN-3361: CapacityScheduler side changes to support non-exclusive node labels.
YARN-3362: Add node label usage in RM CapacityScheduler web UI.
YARN-3365: Enhanced NodeManager to support using the 'tc' tool via container-executor for outbound network traffic control.
YARN-3366: Enhanced NodeManager to support classifying/shaping outgoing network bandwidth traffic originating from YARN containers.
YARN-3383: AdminService should use "warn" instead of "info" to log exception when operation fails.
YARN-3387: Previous AM's container completed status couldn't pass to current AM if AM and RM restarted during the same time.
YARN-3394: Enrich WebApplication proxy documentation.
YARN-3397: YARN rmadmin should skip -failover.
YARN-3404: Display queue name on application page.
YARN-3406: Display count of running containers in the RM's Web UI.
YARN-3410: YARN admin should be able to remove individual application records from RMStateStore.
YARN-3413: Changed Nodelabel attributes (like exclusivity) to be settable only via addToClusterNodeLabels but not changeable at runtime.
YARN-3425: NPE from RMNodeLabelsManager.serviceStop when NodeLabelsManager.serviceInit failed.
YARN-3433: Jersey tests failing with Port in Use -again. .
YARN-3435: AM container to be allocated Appattempt AM container shown as null.
YARN-3443: Create a 'ResourceHandler' subsystem to ease addition of support for new resource types on the NM.
YARN-3448: Added a rolling time-to-live LevelDB timeline store implementation.
YARN-3451: Display attempt start time and elapsed time on the web UI.
YARN-3459: Fix failure of TestLog4jWarningErrorMetricsAppender.
YARN-3463: Integrate OrderingPolicy Framework with CapacityScheduler.
YARN-3480: Remove attempts that are beyond max-attempt limit from state store.
YARN-3494: Expose AM resource limit and usage in CS QueueMetrics.
YARN-3503: Expose disk utilization percentage and bad local and log dir counts in NM metrics.
YARN-3505: Node's Log Aggregation Report with SUCCEED should not cached in RMApps.
YARN-3511: Add errors and warnings page to ATS.
YARN-3517: RM web UI for dumping scheduler logs should be for admins only.
YARN-3521: Support return structured NodeLabel objects in REST API.
YARN-3530: ATS throws exception on trying to filter results without otherinfo.
YARN-3541: Add version info on timeline service / generic history web UI and REST API.
YARN-3543: ApplicationReport should be able to tell whether the Application is AM managed or not.
YARN-3552: RM Web UI shows -1 running containers for completed apps.
YARN-3565: NodeHeartbeatRequest/RegisterNodeManagerRequest should use NodeLabel object instead of String.
YARN-3579: CommonNodeLabelsManager should support NodeLabel instead of string label name when getting node-to-label/label-to-label mappings.
YARN-3580: [JDK8] TestClientRMService.testGetLabelsToNodes fails.
YARN-3581: Deprecate -directlyAccessNodeLabelStore in RMAdminCLI.
YARN-3583: Support of NodeLabel object instead of plain String in YarnClient side.
YARN-3593: Add label-type and Improve "DEFAULT_PARTITION" in Node Labels Page.
YARN-3600: AM container link is broken (on a killed application, at least).
YARN-3604: removeApplication in ZKRMStateStore should also disable watch.
YARN-3632: Ordering policy should be allowed to reorder an application when demand changes.
YARN-3635: Refactored current queue mapping implementation in CapacityScheduler to use a generic PlacementManager framework.
YARN-3654: ContainerLogsPage web UI should not have meta-refresh.
YARN-3695: ServerProxy (NMProxy, etc.) shouldn't retry forever for non network exception.
YARN-3700: Made generic history service load a number of latest applications according to the parameter or the configuration.
YARN-3707: RM Web UI queue filter doesn't work.
YARN-3717: Expose app/am/queue's node-label-expression to RM web UI / CLI / REST-API.
YARN-3727: For better error recovery, check if the directory exists before using it for localization.
YARN-3740: Fixed the typo in the configuration name: APPLICATION_HISTORY_PREFIX_MAX_APPS.
YARN-3751: Fixed AppInfo to check if used resources are null.
YARN-3766: Fixed the apps table column error of generic history web UI.
YARN-3769: Consider user limit when calculating total pending resource for preemption policy in Capacity Scheduler.
YARN-3785: Support for Resource as an argument during submitApp call in MockRM test class.
YARN-3846: RM Web UI queue filter is not working for sub queue.
YARN-3849: Too much of preemption activity causing continuous killing of containers across queues.
YARN-3873: PendingApplications in LeafQueue should also use OrderingPolicy.
YARN-3885: ProportionalCapacityPreemptionPolicy doesn't preempt if queue is more than 2 level.
YARN-3894: RM startup should fail for wrong CS xml NodeLabel capacity configuration.
YARN-3896: RMNode transitioned from RUNNING to REBOOTED because its response id has not been reset synchronously.
YARN-3932: SchedulerApplicationAttempt#getResourceUsageReport and UserInfo should based on total-used-resources.
YARN-3941: Proportional Preemption policy should try to avoid sending duplicate PREEMPT_CONTAINER event to scheduler.
YARN-3946: Update exact reason as to why a submitted app is in ACCEPTED state to app's diagnostic message.
YARN-3948: Display Application Priority in RM Web UI.
YARN-3961: Expose pending, running and reserved containers of a queue in REST API and YARN top.
YARN-3966: Fix excessive loggings in CapacityScheduler.
YARN-3971: Skip RMNodeLabelsManager#checkRemoveFromClusterNodeLabelsOfQueue on nodelabel recovery.
YARN-3978: Configurably turn off the saving of container info in Generic AHS.
YARN-3983: Refactored CapacityScheduleri#FiCaSchedulerApp to easier extend container allocation logic.
YARN-3986: getTransferredContainers in AbstractYarnScheduler should be present in YarnScheduler interface.
YARN-3987: AM container complete msg ack to NM once RM receive it.
YARN-4000: RM crashes with NPE if leaf queue becomes parent queue during restart.
YARN-4023: Publish Application Priority to TimelineServer.
YARN-4026: Refactored ContainerAllocator to accept a list of priories rather than a single priority.
YARN-4082: Container shouldn't be killed when node's label updated.
YARN-4087: Followup fixes after YARN-2019 regarding RM behavior when state-store error occurs.
YARN-4092: Fixed UI redirection to print useful messages when both RMs are in standby mode.
YARN-4101: RM should print alert messages if ZooKeeper and Resourcemanager gets connection issue.
YARN-4108: CapacityScheduler: Improve preemption to only kill containers that would satisfy the incoming request.
YARN-4140: RM container allocation delayed in case of app submitted to Nodelabel partition.
YARN-4149: YARN logs -am should provide an option to fetch all the log files.
YARN-4155: TestLogAggregationService.testLogAggregationServiceWithInterval failing.
YARN-4162: CapacityScheduler: Add resource usage by partition and queue capacity by partition to REST API.
YARN-4215: RMNodeLabels Manager Need to verify and replace node labels for the only modified Node Label Mappings in the request.
YARN-4225: Add preemption status to yarn queue -status for capacity scheduler.
YARN-4233: YARN Timeline Service plugin: ATS v1.5.
YARN-4285: Display resource usage as percentage of queue and cluster in the RM UI.
YARN-4287: Capacity Scheduler: Rack Locality improvement.
YARN-4288: Fixed RMProxy to retry on IOException from local host.
YARN-4290: Add -showDetails option to YARN Nodes CLI to print all nodes reports information.
YARN-4304: AM max resource configuration per partition to be displayed/updated correctly in UI and in various partition related metrics.
YARN-4309: Add container launch related debug information to container logs when a container fails.
YARN-4313: Race condition in MiniMRYarnCluster when getting history server address.
YARN-4315: NaN in Queue percentage for cluster apps page.
YARN-4345: YARN rmadmin -updateNodeResource doesn't work.
YARN-4347: Resource manager fails with Null pointer exception.
YARN-4349: YARN_APPLICATION call to ATS does not have YARN_APPLICATION_CALLER_CONTEXT.
YARN-4365: FileSystemNodeLabelStore should check for root dir existence on startup.
YARN-4384: updateNodeResource CLI should not accept negative values for resource.
YARN-4390: Do surgical preemption based on reserved container in CapacityScheduler.
YARN-4405: Support node label store in non-appendable file system.
YARN-4414: Nodemanager connection errors are retried at multiple levels.
YARN-4416: Deadlock due to synchronised get Methods in AbstractCSQueue.
YARN-4417: Make RM and Timeline-server REST APIs more consistent.
YARN-4418: AM Resource Limit per partition can be updated to ResourceUsage as well.
YARN-4422: Generic AHS sometimes doesn't show started, node, or logs on App page.
YARN-4428: Redirect RM page to AHS page when AHS turned on and RM page is not available.
YARN-4439: Clarify NMContainerStatus#toString method.
YARN-4454: NM to nodelabel mapping going wrong after RM restart.
YARN-4479: Change CS LeafQueue pendingOrderingPolicy to honor recovered apps.
YARN-4497: RM might fail to restart when recovering apps whose attempts are missing.
YARN-4502: Fix two AM containers get allocated when AM restart.
YARN-4522: Queue acl can be checked at app submission.
YARN-4535: Fix checkstyle error in CapacityScheduler.java.
YARN-4546: ResourceManager crash due to scheduling opportunity overflow.
YARN-4557: Fix improper Queues sorting in PartitionedQueueComparator when accessible-node-labels=*.
YARN-4565: Sometimes when sizeBasedWeight FairOrderingPolicy is enabled, under stress appears that cluster is virtually in deadlock.
YARN-4573: Fix test failure in TestRMAppTransitions#testAppRunningKill and testAppKilledKilled.
YARN-4584: RM startup failure when AM attempts greater than max-attempts.
YARN-4598: Invalid event: RESOURCE_FAILED at CONTAINER_CLEANEDUP_AFTER_KILL.
YARN-4607: Pagination support for AppAttempt page TotalOutstandingResource Requests table.
YARN-4610: Reservations continue looking for one app causes other apps to starve.
YARN-4617: LeafQueue#pendingOrderingPolicy should always use fixed ordering policy instead of using same as active applications ordering policy.
YARN-4623: TestSystemMetricsPublisher#testPublishAppAttemptMetricsForUnmanagedAM fails with NPE. 7.
YARN-4625: Make ApplicationSubmissionContext and ApplicationSubmissionContextInfo more consistent.
YARN-4633: TestRMRestart.testRMRestartAfterPreemption fails intermittently.
YARN-4634: Scheduler UI/Metrics need to consider cases like non-queue label mappings.
YARN-4667: RM Admin CLI for refreshNodesResources throws NPE when nothing is configured.
YARN-4696: TimelineClient to add flush operation for deterministic writes (including testing) and Changes to EntityGroupFSTimelineStore for testability.
YARN-4699: Scheduler UI and REST o/p is not in sync when -replaceLabelsOnNode is used to change label of a node.
YARN-4709: NMWebServices produces incorrect JSON for containers.
YARN-4723: NodesListManager$UnknownNodeId ClassCastException.
YARN-4737: Add CSRF filter support in YARN.
YARN-4746: YARN web services should convert parse failures of appId, appAttemptId and containerId to 400.
YARN-4769: Add support for CSRF header in the dump capacity scheduler logs and kill app buttons in RM web UI.
YARN-4785: Inconsistent value type of the "type" field for LeafQueueInfo in response of RM REST API.
YARN-4811: Generate histograms in ContainerMetrics for actual container resource usage.
YARN-4814: ATS 1.5 timelineclient impl call flush after every event write.
YARN-4815: ATS 1.5 timelineclinet impl try to create attempt directory for every event call.
YARN-4822: Refactor existing Preemption Policy of CS for easier adding new approach to select preemption candidates.
YARN-4832: NM side resource value should get updated if change applied in RM side.
YARN-4844: Add getMemorySize/getVirtualCoresSize to o.a.h.y.api.records.Resource.
YARN-4846: Fix random failures for TestCapacitySchedulerPreemption #testPreemptionPolicyShouldRespectAlreadyMarkedKillableContainers.
YARN-4865: Track Reserved resources in ResourceUsage and QueueCapacities.
YARN-4890: Unit test intermittent failure: TestNodeLabelContainerAllocation#testQueueUsedCapacitiesUpdate.
YARN-4893: Fix intermittent failures in TestRMAdminService.
YARN-4916: TestNMProxy.tesNMProxyRPCRetry fails.
YARN-4934: Reserved Resource for QueueMetrics needs to be handled correctly in few cases.
YARN-4955: Add retry for SocketTimeoutException in TimelineClient.
YARN-4984: LogAggregationService shouldn't swallow exception in handling createAppDir() which cause thread leak.
YARN-5023: TestAMRestart#testShouldNotCountFailureToMaxAttemptRetry random failure.
YARN-5048: DelegationTokenRenewer#skipTokenRenewal may throw NPE.
YARN-5076: YARN web interfaces lack XFS (Cross-Frame Script) protection.
YARN-5112: Excessive log warnings for directory permission issue on NM recovery.
YARN-5131: Distributed shell AM fails when extra container arrives during finishing.
YARN-5190: Registering/unregistering container metrics triggered by ContainerEvent and ContainersMonitorEvent are conflict which cause uncaught exception in ContainerMonitorImpl.
YARN-5214: Fixed locking in DirectoryCollection to avoid hanging NMs when various code-paths hit slow disks.
YARN-5270: Solve miscellaneous issues caused by YARN-4844.
YARN-5294: Pass remote ip address down to YarnAuthorizationProvider.
YARN-5296: NMs going OutOfMemory because ContainerMetrics leak in ContainerMonitorImpl.
YARN-5297: Avoid printing a stack trace when recovering an app after the RM restarts.
YARN-5337: Fix Dshell OOM issues.