Ozone

You can provision an Ozone cluster for optimal performance based on the desired storage capacity. Ozone supports up to 400 TB per datanode in Cloudera Runtime 7.1.7 or later.

Table 1. Sizing for 10 PB - 150 TB per DN (low density)The following table provides the sizing information for a low density storage deployment.
Node Type	Service	Storage	Memory	CPU Cores	Network
Master Nodes (3 replicas)	Ozone Manager	*2 2TB NVMe SSD** (exclusive) (one for OM Raft Log and one for RocksDB)	Minimum 128 GB Host Memory 31GB - OM Heap 31 GB - SCM Heap	32	50 Gbps/ 5 GBps
Master Nodes (3 replicas)	Storage Container Manager	*2 1TB NVMe or SATA SSD** (exclusive) (one for SCM Raft Log and one for RocksDB)		32
Worker Nodes (Up to 200)	Ozone Datanode	*1 2TB SATA or NVMe SSD (replication logs) [Optional - In the absence of SSD, can have HDD] 12 to 16 * 8TB HDDs** (user data)	Minimum 128 GB Host Memory DN Java Heap - 16 GB DN off-heap - 15 GB 31 GB - S3G Heap	32
Worker Nodes (Up to 200)	S3 Gateway	NA	31 GB	24
Admin Dashboard (1 replica)	Recon Server	*1 1TB SATA or NVMe SSD**	32 GB - Host memory 16 GB - Recon Heap	16

note

Keyspace: The above configuration can support up to 10B keys because of the 4 TB NVMe on the master nodes.
Network: The network between the datanodes and the compute nodes cannot be oversubscribed by more than 2:1. Networking is sized to support the full (real world) bandwidth of the drives across the network. More drives require faster networks, both at the server level and the switch level.
NVMe:
- NVMe come in RAID1 pairs to provide business continuity for Ozone metadata in case of hardware failure. This is not needed on pure compute nodes which only use them for caching.
- The masters and storage datanodes use NVMe to store Ozone metadata.
- The compute nodes use NVMe for shuffle (Spark, MapReduce, Tez) and for caching.
- The mixed compute datanodes use NVMe for both Ozone metadata and shuffle (Spark, MapReduce, Tez) plus caching.
- Mount Ozone partitions across both drives as RAID1 (800GB), with the remaining space used for shuffle or cache as independant JBOD partitions.
- RAID can be done in hardware or in software
The values are minimums and can go higher depending on requirements
Minimum configuration to start is 3 master nodes and 9 datanodes. This will support erasure coding rs(6,3) in the future. Additional datanodes can be added in increments of 1 to increase storage.

Table 2. Sizing for 10 PB - 400 TB per DN (high density)The following table provides the sizing information for a high density storage deployment.
Node Type	Service	Storage	Memory	CPU Cores	Network
Master Nodes (3 replicas)	Ozone Manager (3x)	*2 2TB NVMe SSD**	Minimum 256 GB Host Memory 31 GB - OM Heap 31 GB - SCM Heap	32	100 GBps/ 10 GBps
Master Nodes (3 replicas)	Storage Container Manager (3x)	*2 1TB NVMe or SATA SSD**		32
Worker Nodes (Up to 200)	Ozone Datanode	*1 2TB SATA or NVMe SSD (replication logs) 24 * 16TB HDDs; or 48 * 8TB HDDs** (user data)	Minimum 128 GB Host Memory DN Java Heap - 16 GB DN off-heap - 15 GB 31 GB - S3G Heap	32
Worker Nodes (Up to 200)	S3 Gateway	NA	31 GB	24
Admin Dashboard (1 replica)	Recon Server	*1 1TB SATA or NVMe SSD**	32 GB - Host memory 16 GB - Recon Heap	16