User:Pokechu22/Chunk Format - Revision history

imported>Pokechu22: Redirect old article

2017-06-12T14:52:40Z

Redirect old article

imported>Pokechu22: /* Tips */ entity

2017-06-12T14:44:22Z

Tips: entity

← Older revision		Revision as of 14:44, 12 June 2017
Line 242:		Line 242:
	* Servers do <em>not</em> need to implement the palette initially (instead always using 13 bits per block), although it is an important optimization later on.		* Servers do <em>not</em> need to implement the palette initially (instead always using 13 bits per block), although it is an important optimization later on.
	* The Notchian server implementation does not send values that are out of bounds for the palette. If such a value is received, the format is being parsed incorrectly. In particular, if you're reading a number with all bits set (15, 31, etc), then you're probably reading sky light data.		* The Notchian server implementation does not send values that are out of bounds for the palette. If such a value is received, the format is being parsed incorrectly. In particular, if you're reading a number with all bits set (15, 31, etc), then you're probably reading sky light data.
	* The number of longs needed for the data array can be calculated as ((16×16×16 blocks)×Bits per block)&~~div~~;64 bits per long (which simplifies to 64×Bits per block). For instance, 13 bits per block requires 832 longs.		* The number of longs needed for the data array can be calculated as ((16×16×16 blocks)×Bits per block)÷64 bits per long (which simplifies to 64×Bits per block). For instance, 13 bits per block requires 832 longs.

	== Sample implementations ==		== Sample implementations ==

imported>Pokechu22: /* Tips */ Number of longs needed

2017-06-12T14:43:58Z

Tips: Number of longs needed

← Older revision		Revision as of 14:43, 12 June 2017
Line 241:		Line 241:
	* The <code>13</code> value for full bits per block is likely to change in the future, so it should not be hardcoded (instead, it should either be calculated or left as a constant).		* The <code>13</code> value for full bits per block is likely to change in the future, so it should not be hardcoded (instead, it should either be calculated or left as a constant).
	* Servers do <em>not</em> need to implement the palette initially (instead always using 13 bits per block), although it is an important optimization later on.		* Servers do <em>not</em> need to implement the palette initially (instead always using 13 bits per block), although it is an important optimization later on.
	* The Notchian server implementation does not send values that are out of bounds for the palette. If such a value is received, the format is being parsed incorrectly.		* The Notchian server implementation does not send values that are out of bounds for the palette. If such a value is received, the format is being parsed incorrectly. In particular, if you're reading a number with all bits set (15, 31, etc), then you're probably reading sky light data.
			* The number of longs needed for the data array can be calculated as ((16×16×16 blocks)×Bits per block)&div;64 bits per long (which simplifies to 64×Bits per block). For instance, 13 bits per block requires 832 longs.

	== Sample implementations ==		== Sample implementations ==

imported>Pokechu22: /* Tips */ Who is motch?

2017-06-06T19:10:27Z

Tips: Who is motch?

← Older revision		Revision as of 19:10, 6 June 2017
Line 241:		Line 241:
	* The <code>13</code> value for full bits per block is likely to change in the future, so it should not be hardcoded (instead, it should either be calculated or left as a constant).		* The <code>13</code> value for full bits per block is likely to change in the future, so it should not be hardcoded (instead, it should either be calculated or left as a constant).
	* Servers do <em>not</em> need to implement the palette initially (instead always using 13 bits per block), although it is an important optimization later on.		* Servers do <em>not</em> need to implement the palette initially (instead always using 13 bits per block), although it is an important optimization later on.
	* The ~~motchian~~ server implementation does not send values that are out of bounds for the palette. If such a value is received, the format is being parsed incorrectly.		* The Notchian server implementation does not send values that are out of bounds for the palette. If such a value is received, the format is being parsed incorrectly.

	== Sample implementations ==		== Sample implementations ==

imported>Pokechu22: Clarify packing of light arrays, and fix a broken anchor

2017-04-10T21:55:06Z

Clarify packing of light arrays, and fix a broken anchor

← Older revision		Revision as of 21:55, 10 April 2017
Line 143:		Line 143:
	\|-		\|-
	\| Data		\| Data
	\| Array of [[#Chunk Section\|Chunk Section]]		\| Array of [[#Chunk Section structure\|Chunk Section]]
	\| The length of the array is equal to the number of bits set in Primary Bit Mask. Sections are sent bottom-to-top, i.e. the first section, if sent, extends from Y=0 to Y=15.		\| The length of the array is equal to the number of bits set in Primary Bit Mask. Sections are sent bottom-to-top, i.e. the first section, if sent, extends from Y=0 to Y=15.
	\|-		\|-
Line 151:		Line 151:
	\|}		\|}

	=== Chunk Section ===		=== Chunk Section structure ===

	A Chunk Section is defined in terms of other [[data types]]. A Chunk Section consists of the following fields:		A Chunk Section is defined in terms of other [[data types]]. A Chunk Section consists of the following fields:
Line 190:		Line 190:
	\|}		\|}

	~~In half-byte arrays, two values are packed into each byte. Even-indexed items are packed into the ''low bits'', odd-indexed into the ''high bits''.~~		Data Array, Block Light, and Sky Light are given for each block with increasing x coordinates, within rows of increasing z coordinates, within layers of increasing y coordinates. For the half-byte light arrays, even-indexed items (those with an even x coordinate, starting at 0) are packed into the ''low bits'', odd-indexed into the ''high bits''.

	Data Array, Block Light, and Sky Light are given for each block with increasing x coordinates, within rows of increasing z coordinates, within layers of increasing y coordinates.

	The Data Array, although varying in length, will never be padded due to the number of blocks being evenly divisible by 64, which is the number of bits in a long.		The Data Array, although varying in length, will never be padded due to the number of blocks being evenly divisible by 64, which is the number of bits in a long.

imported>Pokechu22: s/notchain/notchian/

2017-03-28T21:59:39Z

s/notchain/notchian/

imported>Pokechu22: Adjust word-wrapping

2017-03-23T17:28:48Z

Adjust word-wrapping

← Older revision		Revision as of 17:28, 23 March 2017
Line 25:		Line 25:
	It is important to note that a chunk composed entirely of empty sections is different from an empty (ie, unloaded) chunk column. When a block is changed in an empty section, the section is created (as all air), and the block is set. When a block is changed in an empty chunk, the behavior is undefined (but generally, nothing happens).		It is important to note that a chunk composed entirely of empty sections is different from an empty (ie, unloaded) chunk column. When a block is changed in an empty section, the section is created (as all air), and the block is set. When a block is changed in an empty chunk, the behavior is undefined (but generally, nothing happens).

	The '''primary bit mask''' simply determines which sections are being sent. The least significant bit is for the lowest section (y=0 to y=15). Only 16 bits can be set in it (with the 16th bit controlling the y=240 to y=255 section); sections above y=255 are not valid for the notchain client. To check whether a section is included, use <~~code~~>((mask & (1 << sectionY)) != 0)</~~code~~>.		The '''primary bit mask''' simply determines which sections are being sent. The least significant bit is for the lowest section (y=0 to y=15). Only 16 bits can be set in it (with the 16th bit controlling the y=240 to y=255 section); sections above y=255 are not valid for the notchain client. To check whether a section is included, use <span style="white-space:nowrap"><syntaxhighlight lang="c" inline>((mask & (1 << sectionY)) != 0)</syntaxhighlight></span>.

	=== Global and section palettes ===		=== Global and section palettes ===
Line 35:		Line 35:
	There are 2 palettes that are used in the game: the global palette and the section palette.		There are 2 palettes that are used in the game: the global palette and the section palette.

	{{Anchor\|Global palette}}The '''global palette''' is the standard mapping of IDs to block states. Currently, it is a combination Block ID and Metadata <span style="white-space:nowrap">(<syntaxhighlight lang="c" inline>(blockId << 4) \| metadata</syntaxhighlight>)</span>. Note that thus, the global palette is not continuous<ref group="concept note">The global palette is not continuous in more ways than 1. The more obvious manner is that not all blocks have metadata: for instance, dirt (ID 3) has only 3 states (dirt, coarse dirt, and podzol), so the palette surrounding it is <code>000000011 0000; 000000011 0001; 000000011 0010; 000000100 0000</code>. The second way is that structure blocks have an ID of 255, even though there is currently no block with ID 254; thus, there is a large gap.</ref>. Entries not defined within the global palette are treated as air (even if the block ID itself is known, if the metadata is not known, the state is treated as air). Note that the global palette is currently represented by 13 bits per entry<ref group="concept note">The number of bits in the global palette via the ceil of a base-2 logarithm of the highest value in the palette.</ref>, with 9 bits for the block ID and 4 bits for the metadata.		{{Anchor\|Global palette}}The '''global palette''' is the standard mapping of IDs to block states. Currently, it is a combination Block ID and Metadata <span style="white-space:nowrap">(<syntaxhighlight lang="c" inline>(blockId << 4) \| metadata</syntaxhighlight>)</span>. Note that thus, the global palette is not continuous<ref group="concept note">The global palette is not continuous in more ways than 1. The more obvious manner is that not all blocks have metadata: for instance, dirt (ID 3) has only 3 states (dirt, coarse dirt, and podzol), so the palette surrounding it is <code>000000011 0000; 000000011 0001; 000000011 0010; 000000100 0000</code>. The second way is that structure blocks have an ID of 255, even though there is currently no block with ID 254; thus, there is a large gap.</ref>. Entries not defined within the global palette are treated as air (even if the block ID itself is known, if the metadata is not known, the state is treated as air). Note that the global palette is currently represented by 13 bits per entry<ref group="concept note">The number of bits in the global palette via the ceil of a base-2 logarithm of the highest value in the palette.</ref>, with 9 bits for the block ID and 4 bits for the metadata.

	The basic implementation looks like this:		The basic implementation looks like this:
Line 202:		Line 202:
	* 9 and above: The palette is not sent. Blocks are encoded by their whole ID in the global palette, with bits per block being set as the base 2 logarithm of the number of block states, rounded up. For the current vanilla release, this is 13 bits per block.		* 9 and above: The palette is not sent. Blocks are encoded by their whole ID in the global palette, with bits per block being set as the base 2 logarithm of the number of block states, rounded up. For the current vanilla release, this is 13 bits per block.

	The global palette encodes a block as 13 bits. It uses the {{Minecraft Wiki\|Data values#Block IDs\|block ID}} for the first 9 bits, and the block damage value for the last 4 bits. For example, Diorite (block ID <code>1</code> for <code>minecraft:stone</code> with damage <code>3</code>) would be encoded as <code>000000001 0011</code>. If a block is not found in the global palette (either due to not having a valid damage value or due to not being a valid ID), it will be treated as air.		The global palette encodes a block as 13 bits. It uses the {{Minecraft Wiki\|Data values#Block IDs\|block ID}} for the first 9 bits, and the block damage value for the last 4 bits. For example, Diorite (block ID <code>1</code> for <code>minecraft:stone</code> with damage <code>3</code>) would be encoded as <code>000000001 0011</code>. If a block is not found in the global palette (either due to not having a valid damage value or due to not being a valid ID), it will be treated as air.

	If Minecraft Forge is installed and a sufficiently large number of blocks are added, the bits per block value for the global palette will be increased to compensate for the increased ID count. This increase can go up to 16 bits per block (for a total of 4096 block IDs; when combined with the 16 damage values, there are 65536 total states). You can get the number of blocks with the "Number of ids" field found in the [[Minecraft Forge Handshake#RegistryData\|RegistryData packet in the Forge Handshake]].		If Minecraft Forge is installed and a sufficiently large number of blocks are added, the bits per block value for the global palette will be increased to compensate for the increased ID count. This increase can go up to 16 bits per block (for a total of 4096 block IDs; when combined with the 16 damage values, there are 65536 total states). You can get the number of blocks with the "Number of ids" field found in the [[Minecraft Forge Handshake#RegistryData\|RegistryData packet in the Forge Handshake]].

imported>Pokechu22: More info on empty sections and the primary bit mask

2017-03-23T15:52:11Z

More info on empty sections and the primary bit mask

← Older revision		Revision as of 15:52, 23 March 2017
Line 13:		Line 13:
	Chunk columns store biomes, block entities, entities, tick data, and an array of sections.		Chunk columns store biomes, block entities, entities, tick data, and an array of sections.

	{{Anchor\|Chunk section}}A '''chunk section''' is a 16×16×16 collection of blocks (chunk sections are cubic). This is the actual area that blocks are stored in, and is often the concept Mojang refers to via "chunk". Breaking columns into sections wouldn't be useful, except that you don't need to send all chunk sections in a column: If a section is empty<ref group="concept note">Empty is defined in vanilla as being composed of all air, but this can result in lighting issues ([https://bugs.mojang.com/browse/MC-80966 MC-80966]). Custom servers should consider defining empty to mean something like "completely air and without lighting data" or "completely air and with no blocks in the neighboring sections that need to be lit by light from this section".</ref>, then it doesn't need to be sent~~. For the average world,~~ this means you don't need to send approximately 60% of the world's data, since it's all air. A chunk section can contain at maximum 4096 (16×16×16, or 2<sup>12</sup>) unique IDs (but, it is highly unlikely that such a section will occur in normal circumstances).		{{Anchor\|Chunk section}}A '''chunk section''' is a 16×16×16 collection of blocks (chunk sections are cubic). This is the actual area that blocks are stored in, and is often the concept Mojang refers to via "chunk". Breaking columns into sections wouldn't be useful, except that you don't need to send all chunk sections in a column: If a section is empty, then it doesn't need to be sent (more on this later).

	Chunk sections store blocks and light data (both block light and sky light). Additionally, they can be associated with a [[#Section palette\|section palette]].		Chunk sections store blocks and light data (both block light and sky light). Additionally, they can be associated with a [[#Section palette\|section palette]]. A chunk section can contain at maximum 4096 (16×16×16, or 2<sup>12</sup>) unique IDs (but, it is highly unlikely that such a section will occur in normal circumstances).

	Chunk columns and chunk sections are both displayed when chunk border rendering is enabled (<kbd>F3</kbd>+<kbd>G</kbd>). Chunk columns borders are indicated via the red vertical lines, while chunk sections borders are indicated by the blue lines.		Chunk columns and chunk sections are both displayed when chunk border rendering is enabled (<kbd>F3</kbd>+<kbd>G</kbd>). Chunk columns borders are indicated via the red vertical lines, while chunk sections borders are indicated by the blue lines.

			=== Empty sections and the primary bit mask ===

			As previously mentioned, chunk sections can be '''empty'''. Sections which contain no useful data are treated as empty<ref group="concept note">Empty is defined by the notchian server as being composed of all air, but this can result in lighting issues ([https://bugs.mojang.com/browse/MC-80966 MC-80966]). Custom servers should consider defining empty to mean something like "completely air and without lighting data" or "completely air and with no blocks in the neighboring sections that need to be lit by light from this section".</ref>, and are not sent to the client, as the client is able to infer the contents<ref group="concept note">Generally meaning, "it's all air". Of course, lighting is an issue as with before - the notchain client assumes 0 block light and 15 sky light, even when that's not valid (underground sections shouldn't be skylit, and sections near light sources should be lit).</ref>. For the average world, this means around 60% of the world's data doesn't need to be sent, since it's all air; this is a significant save.

			It is important to note that a chunk composed entirely of empty sections is different from an empty (ie, unloaded) chunk column. When a block is changed in an empty section, the section is created (as all air), and the block is set. When a block is changed in an empty chunk, the behavior is undefined (but generally, nothing happens).

			The '''primary bit mask''' simply determines which sections are being sent. The least significant bit is for the lowest section (y=0 to y=15). Only 16 bits can be set in it (with the 16th bit controlling the y=240 to y=255 section); sections above y=255 are not valid for the notchain client. To check whether a section is included, use <code>((mask & (1 << sectionY)) != 0)</code>.

	=== Global and section palettes ===		=== Global and section palettes ===
Line 33:		Line 41:
	<syntaxhighlight lang="java">		<syntaxhighlight lang="java">
	long getGlobalPaletteIDFromState(BlockState state) {		long getGlobalPaletteIDFromState(BlockState state) {
	if (state.isValid()) {		if (state.isValid()) {
	return (state.getId() << 4) \| state.getMetadata();		return (state.getId() << 4) \| state.getMetadata();
	} else {		} else {
	return 0;		return 0;
	}		}
	}		}

	BlockState getStateFromGlobalPaletteID(long id) {		BlockState getStateFromGlobalPaletteID(long id) {
	int blockID = (id >> 4);		int blockID = (id >> 4);
	byte metadata = (id & 0x0F);		byte metadata = (id & 0x0F);
	BlockState state = new BlockState(blockID, metadata);		BlockState state = new BlockState(blockID, metadata);
	if (state.isValid()) {		if (state.isValid()) {
	return state;		return state;
	} else {		} else {
	return new BlockState(0, 0); // Air		return new BlockState(0, 0); // Air
	}		}
	}		}
	</syntaxhighlight>		</syntaxhighlight>
Line 62:		Line 70:
	The '''ground-up continuous''' value (tentative name) is one of the more confusing properties of the chunk data packet, simply because there's no good name for it. It controls two different behaviors of the chunk data packet, one that most people need, and one that most don't.		The '''ground-up continuous''' value (tentative name) is one of the more confusing properties of the chunk data packet, simply because there's no good name for it. It controls two different behaviors of the chunk data packet, one that most people need, and one that most don't.

	When ground-up continuous is set, the chunk data packet is used to create a ''new'' chunk. This includes biome data, and all (non-empty) sections in the chunk.		When ground-up continuous is set, the chunk data packet is used to create a ''new'' chunk. This includes biome data, and all (non-empty) sections in the chunk. Sections not specified in the primary bit mask are empty sections.

	{{Warning2\|Sending a packet with ground-up continuous enabled over a chunk that already exists will '''leak memory''' clientside.		{{Warning2\|Sending a packet with ground-up continuous enabled over a chunk that already exists will '''leak memory''' clientside.
Line 70:		Line 78:
	The <code>MultiplayerChunkCache</code> values in F3 show the number of chunks in the client's 2 chunk storage mechanisms; if the numbers aren't equal, you've leaked chunks.}}		The <code>MultiplayerChunkCache</code> values in F3 show the number of chunks in the client's 2 chunk storage mechanisms; if the numbers aren't equal, you've leaked chunks.}}

	When ground-up continuous is not set, then the chunk data packet acts as a large [[Protocol#Multi Block Change\|Multi Block Change]] packet, changing all of the blocks in the given section at once. This can have some performance benefits, especially for lighting purposes. BIome data is ''not'' sent when ground-up continuous is not set; that means that biomes can't be changed once a chunk is loaded.		When ground-up continuous is not set, then the chunk data packet acts as a large [[Protocol#Multi Block Change\|Multi Block Change]] packet, changing all of the blocks in the given section at once. This can have some performance benefits, especially for lighting purposes. BIome data is ''not'' sent when ground-up continuous is not set; that means that biomes can't be changed once a chunk is loaded. Sections not specified in the primary bit mask are not changed and should be left as-is.

	{{Warning2\|As with [[Protocol#Multi Block Change\|Multi Block Change]] and [[Protocol#Block Change\|Block Change]], it is not safe to send this packet in unloaded chunks, as it can corrupt notchian client's shared empty chunk. Clients should ''ignore'' such packets, and servers should not send non-ground-up continuous chunk data packets into unloaded chunks.}}		{{Warning2\|As with [[Protocol#Multi Block Change\|Multi Block Change]] and [[Protocol#Block Change\|Block Change]], it is not safe to send this packet in unloaded chunks, as it can corrupt notchian client's shared empty chunk. Clients should ''ignore'' such packets, and servers should not send non-ground-up continuous chunk data packets into unloaded chunks.}}
Line 271:		Line 279:
	private void ReadChunkColumn(Chunk chunk, bool full, int mask, Buffer data) {		private void ReadChunkColumn(Chunk chunk, bool full, int mask, Buffer data) {
	for (int sectionY = 0; sectionY < CHUNK_HEIGHT / SECTION_HEIGHT; y++) {		for (int sectionY = 0; sectionY < CHUNK_HEIGHT / SECTION_HEIGHT; y++) {
	if ((mask & (1 << ~~chunkY~~)) != 0) { // Is the given bit set in the mask?		if ((mask & (1 << sectionY)) != 0) { // Is the given bit set in the mask?
	byte bitsPerBlock = ReadByte(data);		byte bitsPerBlock = ReadByte(data);

imported>Pokechu22: Detail ground-up continuous

2017-03-22T19:36:37Z

Detail ground-up continuous

← Older revision		Revision as of 19:36, 22 March 2017
Line 57:		Line 57:

	{{Warning2\|Note that the Notchian client (and server) store their chunk data within the compacted, paletted format. Sending non-compacted data not only wastes bandwidth, but also leads to increased memory use clientside; while this is OK for an initial implementation it is strongly encouraged that one compacts the block data as soon as possible.}}		{{Warning2\|Note that the Notchian client (and server) store their chunk data within the compacted, paletted format. Sending non-compacted data not only wastes bandwidth, but also leads to increased memory use clientside; while this is OK for an initial implementation it is strongly encouraged that one compacts the block data as soon as possible.}}

			=== Ground-up continuous ===

			The '''ground-up continuous''' value (tentative name) is one of the more confusing properties of the chunk data packet, simply because there's no good name for it. It controls two different behaviors of the chunk data packet, one that most people need, and one that most don't.

			When ground-up continuous is set, the chunk data packet is used to create a ''new'' chunk. This includes biome data, and all (non-empty) sections in the chunk.

			{{Warning2\|Sending a packet with ground-up continuous enabled over a chunk that already exists will '''leak memory''' clientside.

			Make sure to unload chunks before overwriting them with the [[Protocol#Unload Chunk\|Unload Chunk]] packet. That packet can always be sent even on unloaded chunks, so in situations where the chunk might or might not be loaded already, it's valid to send it again (but avoid sending it in excess).

			The <code>MultiplayerChunkCache</code> values in F3 show the number of chunks in the client's 2 chunk storage mechanisms; if the numbers aren't equal, you've leaked chunks.}}

			When ground-up continuous is not set, then the chunk data packet acts as a large [[Protocol#Multi Block Change\|Multi Block Change]] packet, changing all of the blocks in the given section at once. This can have some performance benefits, especially for lighting purposes. BIome data is ''not'' sent when ground-up continuous is not set; that means that biomes can't be changed once a chunk is loaded.

			{{Warning2\|As with [[Protocol#Multi Block Change\|Multi Block Change]] and [[Protocol#Block Change\|Block Change]], it is not safe to send this packet in unloaded chunks, as it can corrupt notchian client's shared empty chunk. Clients should ''ignore'' such packets, and servers should not send non-ground-up continuous chunk data packets into unloaded chunks.}}

	=== Notes ===		=== Notes ===
Line 85:		Line 101:
	\| Ground-Up Continuous		\| Ground-Up Continuous
	\| Boolean		\| Boolean
	\| This is true if the packet represents all chunk sections in this vertical chunk column. If true, the chunk that was previously there should be replaced with this chunk. If false, this packet is instead modifying the given chunk sections, but leaves the other sections alone.		\| See [[#Ground-up continuous]]
	\|-		\|-
	\| Primary Bit Mask		\| Primary Bit Mask
Line 120:		Line 136:
	\| Data		\| Data
	\| Array of [[#Chunk Section\|Chunk Section]]		\| Array of [[#Chunk Section\|Chunk Section]]
	\| The length of the array is equal to the number of bits set in Primary Bit Mask. ~~Chunks~~ are sent bottom-to-top, i.e. the first ~~chunk~~, if sent, extends from Y=0 to Y=15.		\| The length of the array is equal to the number of bits set in Primary Bit Mask. Sections are sent bottom-to-top, i.e. the first section, if sent, extends from Y=0 to Y=15.
	\|-		\|-
	\| Biomes		\| Biomes

imported>Pokechu22: /* Sample implementations */ ... I recommended isolating into separate methods, but didn't do so myself; change that.

2017-03-22T17:54:44Z

Sample implementations: ... I recommended isolating into separate methods, but didn't do so myself; change that.

← Older revision		Revision as of 17:54, 22 March 2017
Line 313:		Line 313:
	}		}

	~~byte metadata = data & 0xF;~~		BlockState state = GetStateFromGlobalPaletteID(data);
	~~uint id~~ = data ~~>> 4~~;		section.SetState(x, y, z, state);

	section.~~SetBlock~~(x, y, z, ~~id, metadata~~);
	}		}
	}		}
Line 357:		Line 355:
	}		}
	}		}
			}

			// Value should already have gone through the section palette
			private BlockState GetStateFromGlobalPaletteID(uint value) {
			// This method is subject to change in future MC versions

			byte metadata = data & 0xF;
			uint id = data >> 4;

			return BlockState.ForIDAndMeta(id, metadata);
	}		}
	</syntaxhighlight>		</syntaxhighlight>
Line 426:		Line 434:
	}		}

	~~byte metadata~~ = section.~~GetMetadata(x, y, z);~~		BlockState state = section.GetState(x, y, z);
	~~uint id = section.GetBlockID~~(x, y, z);

	uint value = ~~id << 4 \| metadata~~;		uint value = GetGlobalPaletteIDFromState(state);
	value &= individualValueMask;		value &= individualValueMask;

Line 466:		Line 473:
	}		}
	}		}
			}

			private uint GetGlobalPaletteIDFromState(BlockState state) {
			// NOTE: This method will probably change in new versions
			byte metadata = state.getMetadata();
			uint id = section.GetBlockID(x, y, z);

			return id << 4 \| metadata;
	}		}
	</syntaxhighlight>		</syntaxhighlight>

← Older revision		Revision as of 21:59, 28 March 2017
Line 21:		Line 21:
	=== Empty sections and the primary bit mask ===		=== Empty sections and the primary bit mask ===

	As previously mentioned, chunk sections can be '''empty'''. Sections which contain no useful data are treated as empty<ref group="concept note">Empty is defined by the notchian server as being composed of all air, but this can result in lighting issues ([https://bugs.mojang.com/browse/MC-80966 MC-80966]). Custom servers should consider defining empty to mean something like "completely air and without lighting data" or "completely air and with no blocks in the neighboring sections that need to be lit by light from this section".</ref>, and are not sent to the client, as the client is able to infer the contents<ref group="concept note">Generally meaning, "it's all air". Of course, lighting is an issue as with before - the ~~notchain~~ client assumes 0 block light and 15 sky light, even when that's not valid (underground sections shouldn't be skylit, and sections near light sources should be lit).</ref>. For the average world, this means around 60% of the world's data doesn't need to be sent, since it's all air; this is a significant save.		As previously mentioned, chunk sections can be '''empty'''. Sections which contain no useful data are treated as empty<ref group="concept note">Empty is defined by the notchian server as being composed of all air, but this can result in lighting issues ([https://bugs.mojang.com/browse/MC-80966 MC-80966]). Custom servers should consider defining empty to mean something like "completely air and without lighting data" or "completely air and with no blocks in the neighboring sections that need to be lit by light from this section".</ref>, and are not sent to the client, as the client is able to infer the contents<ref group="concept note">Generally meaning, "it's all air". Of course, lighting is an issue as with before - the notchian client assumes 0 block light and 15 sky light, even when that's not valid (underground sections shouldn't be skylit, and sections near light sources should be lit).</ref>. For the average world, this means around 60% of the world's data doesn't need to be sent, since it's all air; this is a significant save.

	It is important to note that a chunk composed entirely of empty sections is different from an empty (ie, unloaded) chunk column. When a block is changed in an empty section, the section is created (as all air), and the block is set. When a block is changed in an empty chunk, the behavior is undefined (but generally, nothing happens).		It is important to note that a chunk composed entirely of empty sections is different from an empty (ie, unloaded) chunk column. When a block is changed in an empty section, the section is created (as all air), and the block is set. When a block is changed in an empty chunk, the behavior is undefined (but generally, nothing happens).

	The '''primary bit mask''' simply determines which sections are being sent. The least significant bit is for the lowest section (y=0 to y=15). Only 16 bits can be set in it (with the 16th bit controlling the y=240 to y=255 section); sections above y=255 are not valid for the ~~notchain~~ client. To check whether a section is included, use <span style="white-space:nowrap"><syntaxhighlight lang="c" inline>((mask & (1 << sectionY)) != 0)</syntaxhighlight></span>.		The '''primary bit mask''' simply determines which sections are being sent. The least significant bit is for the lowest section (y=0 to y=15). Only 16 bits can be set in it (with the 16th bit controlling the y=240 to y=255 section); sections above y=255 are not valid for the notchian client. To check whether a section is included, use <span style="white-space:nowrap"><syntaxhighlight lang="c" inline>((mask & (1 << sectionY)) != 0)</syntaxhighlight></span>.

	=== Global and section palettes ===		=== Global and section palettes ===
Line 64:		Line 64:
	{{Anchor\|Section palette}}A '''section palette''' is used to map IDs within a [[#Chunk section\|chunk section]] to global palette IDs. Other than skipping empty sections, correct use of the section palette is the biggest place where data can be saved. Given that most sections contain only a few blocks, using 13 bits to represent a chunk section that is only stone, gravel, and air would be extremely wasteful. Instead, a list of IDs are sent mapping indexes to global palette IDs (for instance, <code>0x10 0xD0 0x00</code>), and indexes within the section palette are used (so stone would be sent as <code>0</code>, gravel <code>1</code>, and air <code>2</code>)<ref group="concept note">There is no requirement for IDs in a section palette to be [[Wikipedia:Monotonic\|monotonic]]; the order within the list is entirely arbitrary and often has to deal with how the palette is built (if it finds a stone block before an air block, stone can come first). (However, although the order of the section palette entries can be arbitrary, it can theoretically be optimized to ensure the maximum possible GZIP compression. This optimization offers little to no gain, so generally do not attempt it.) However, there shouldn't be any gaps in the section palette, as gaps would increase the size of the section palette when it is sent.</ref>. The number of bits per ID in the section palette varies from 4 to 8; if fewer than 4 bits would be needed it's increased to 4<ref group="concept note">Most likely, sizes smaller than 4 are not used in the section palette because it would require the palette to be resized several times as it is built in the majority of cases; the processing cost would be higher than the data saved.</ref> and if more than 8 would be needed, the section palette is not used and instead global palette IDs are used<ref group="concept note">Most likely, sizes larger than 8 use the global palette because otherwise, the amount of data used to transmit the palette would exceed the savings that the section palette would grant.</ref>.		{{Anchor\|Section palette}}A '''section palette''' is used to map IDs within a [[#Chunk section\|chunk section]] to global palette IDs. Other than skipping empty sections, correct use of the section palette is the biggest place where data can be saved. Given that most sections contain only a few blocks, using 13 bits to represent a chunk section that is only stone, gravel, and air would be extremely wasteful. Instead, a list of IDs are sent mapping indexes to global palette IDs (for instance, <code>0x10 0xD0 0x00</code>), and indexes within the section palette are used (so stone would be sent as <code>0</code>, gravel <code>1</code>, and air <code>2</code>)<ref group="concept note">There is no requirement for IDs in a section palette to be [[Wikipedia:Monotonic\|monotonic]]; the order within the list is entirely arbitrary and often has to deal with how the palette is built (if it finds a stone block before an air block, stone can come first). (However, although the order of the section palette entries can be arbitrary, it can theoretically be optimized to ensure the maximum possible GZIP compression. This optimization offers little to no gain, so generally do not attempt it.) However, there shouldn't be any gaps in the section palette, as gaps would increase the size of the section palette when it is sent.</ref>. The number of bits per ID in the section palette varies from 4 to 8; if fewer than 4 bits would be needed it's increased to 4<ref group="concept note">Most likely, sizes smaller than 4 are not used in the section palette because it would require the palette to be resized several times as it is built in the majority of cases; the processing cost would be higher than the data saved.</ref> and if more than 8 would be needed, the section palette is not used and instead global palette IDs are used<ref group="concept note">Most likely, sizes larger than 8 use the global palette because otherwise, the amount of data used to transmit the palette would exceed the savings that the section palette would grant.</ref>.

	{{Warning2\|Note that the ~~Notchian~~ client (and server) store their chunk data within the compacted, paletted format. Sending non-compacted data not only wastes bandwidth, but also leads to increased memory use clientside; while this is OK for an initial implementation it is strongly encouraged that one compacts the block data as soon as possible.}}		{{Warning2\|Note that the notchian client (and server) store their chunk data within the compacted, paletted format. Sending non-compacted data not only wastes bandwidth, but also leads to increased memory use clientside; while this is OK for an initial implementation it is strongly encouraged that one compacts the block data as soon as possible.}}

	=== Ground-up continuous ===		=== Ground-up continuous ===
Line 196:		Line 196:
	The Data Array, although varying in length, will never be padded due to the number of blocks being evenly divisible by 64, which is the number of bits in a long.		The Data Array, although varying in length, will never be padded due to the number of blocks being evenly divisible by 64, which is the number of bits in a long.

	There are several values that can be used for the bits per block value. In most cases, invalid values will be interpreted as a different value when parsed by the ~~Notchian~~ client, meaning that chunk data will be parsed incorrectly if you use an invalid bits per block. Servers must make sure that the bits per block value is correct.		There are several values that can be used for the bits per block value. In most cases, invalid values will be interpreted as a different value when parsed by the notchian client, meaning that chunk data will be parsed incorrectly if you use an invalid bits per block. Servers must make sure that the bits per block value is correct.

	* up to 4: Blocks are encoded as 4 bits. The palette is used and sent.		* up to 4: Blocks are encoded as 4 bits. The palette is used and sent.
Line 243:		Line 243:
	* The <code>13</code> value for full bits per block is likely to change in the future, so it should not be hardcoded (instead, it should either be calculated or left as a constant).		* The <code>13</code> value for full bits per block is likely to change in the future, so it should not be hardcoded (instead, it should either be calculated or left as a constant).
	* Servers do <em>not</em> need to implement the palette initially (instead always using 13 bits per block), although it is an important optimization later on.		* Servers do <em>not</em> need to implement the palette initially (instead always using 13 bits per block), although it is an important optimization later on.
	* The ~~Notchain~~ server implementation does not send values that are out of bounds for the palette. If such a value is received, the format is being parsed incorrectly.		* The motchian server implementation does not send values that are out of bounds for the palette. If such a value is received, the format is being parsed incorrectly.

	== Sample implementations ==		== Sample implementations ==