Benchmarks and Measurements

Eyeballed Lookup Table

Before we had access to the raw binary data, we ran benchmarks! Below is our first attempt at measuring bandwidth. It is an eye-measured median of the incoming kB/s from the Networking window in studio after stabilization. The code is on the server and fires a remote with the Value every task.wait().

Value	Median Bandwidth (kB/s)	Notes	Conclusions
noise	±0.02	Noise from environment and other factors
baseline	0.16		Roblox is constantly sending data in the background
()	0.85	Testing if remote event has a cost	Yes
(nil)	0.91	Testing if nil is the same as void	No.
(true), (false), ('')	0.97	Testing if value changes cost	No. Only serialize 2 or more booleans for optimal results
('\0'), ('a'), ('0')	1.03	Testing if value changes cost	No. Past this point, it is better to serialize to strings
('aa'), ('', '')	1.09
(true, false), (false, true), ({true})	1.10	Testing if bit-packed. Testing if order changes cost	No. No
('aaa'), (Vector2int16.new(-32768, 9274))	1.16
(true, true, true), ('aaaa'), ('a', 'a'), ({true, true}), ('', '', '')	1.22	Empty string cost measured	Empty string cost averages 0.12 KB/s. Separating arguments is costly!
('aaaaa'), (Vector3int16.new(-32768, 9274, 32767))	1.28
(true, true, true, true)	1.34
('aaaaaa')	1.35
(Vector2.zero), (Vector2.one), (Vector2.new(-32768, 32767))	1.40	Testing if value changes cost	No
('aaaaaaa'), ('a', 'a', 'a'), (0), (-96495734574.4864)	1.41	Testing if value changes cost	No
(true, true, true, true, true), ('aaaaaaaa')	1.47	Character cost measured	Character cost averages 0.07 KB/s
(true, true, true, true, true, true), ('a', 'a', 'a', 'a')	1.60
(Vector3.zero), (Vector3.one), (Vector3.new(-2402.39, 938403, 2057492.4953))	1.66	Testing if value changes cost	No
(true, true, true, true, true, true, true), (CFrame.identity), (CFrame.new() + Vector3.new(938, 0, 0))	1.73		The CFrame's position is always sent
('a', 'a', 'a', 'a', 'a')	1.79	Cost measured	String cost averages 0.19 KB/s
(true, true, true, true, true, true, true, true)	1.86	Cost measured	Boolean cost averages 0.13 KB/s
(0, 0)	1.98
((CFrame.fromEulerAngles(9, 2, -5.3)), (CFrame.fromEulerAngles(9, 2, -5.3) + Vector3.new(938, 0, 0))	2.10	Investigating if position and rotation are sent separately	The rotation is sent separately, and only if it needs to be sent
(0, 0, 0)	2.54	Dang
(0, 0, 0, 0)	3.10	Cost measured	Number cost averages 0.56 KB/s. Numbers are expensive! Sad

Main takeaways:

• Arguments have overhead! Avoid separating arguments if possible!
• Numbers are expensive! Avoid sending numbers! Even CFrames can be cheaper.
• Booleans are cheap! Don't serialize a single boolean. Serialize 2 or more booleans at a time!
• Strings are cheap! Serialize to strings!
• Nil takes up space!!! Don't send nils for no reason!!!!!!!!

How Can I Trust These Results?

To now compare these results with the binary data above, we can take the difference between two median rates, and compare them with the theoretical difference in bytes. Sending () has a rate of 0.85 kB/s, and sending ('aa') has a rate of 1.09 kB/s. The difference is 0.24 kB/s.

Now to apply theory, the binary suggests that 'aa' uses 1 byte for the type, 1 byte for the length, and 2 bytes for the characters. This means 4 bytes. The rate of these measurements is 60 times a second. This means that the theoretical difference in bytes is 4 Bytes * 60 / second = 240 Bytes / second. This means that the theoretical difference in kB/s is 0.24 kB/s. This is the same as the measured difference in kB/s. This leads us to believe that the binary data is correct, and that the measurements above are accurate as well. This also means the Network Stats window displays Kilobytes instead of Kibibytes.

Proper Benchmarks

Below is our second attempt to measure, using different benchmarks that verifies the above table with better accuracy.

To generate a JSON file of the test below results you can download this place file and run it in studio. For the json file of this benchmark presented on this page you can download this json file. Just be warned, it will take a few hours to complete. The results are measured using the Stats service's DataRecieveKbps property 2000 times every 1-3 frames once the recieve rate has stabilized. It waits for the recieve rate to plateau at 0.15kb/s before starting the test, then waits 10 seconds to allow the averaged Stats.DataReceiveKbps to plateau at the maximum value before starting the measurements. Then it appends the current Stats.DataReceiveKbps value to an array every 1-3 frames 2000 times. The process is repeated for all the different kinds of data, which takes a while.

How Can I Trust These Results?

To compare these results with the binary data above, we can take the difference between two median rates, and compare them with the theoretical difference in bytes. Sending () has a rate of 0.783 kB/s, and sending (7) has a rate of 1.323 kB/s. The difference is 0.540 kB/s.

Now to apply theory, the binary suggests that (7) uses 1 byte for the type and 8 bytes for the number. This means 9 Bytes. The rate of these measurements is 60 times a second. This means that the theoretical difference in bytes is 9 Bytes * 60 / second = 540 Bytes / second. This means that the theoretical difference in kB/s is 0.540 kB/s. This is the same as the measured difference in kB/s. This leads us to believe that the binary data is correct, and that the measurements below are accurate as well. This also means that the Stats service displays Kilobytes instead of Kibibytes.

Results

()

('')

(true)

(false)

({})

('A')

(string.char(255))

('a')

(string.char(0))

({true})

('aa')

(true, false)

({''})

('', '')

(Vector2int16.new(1, -3))

(Vector2int16.new(-1, -3))

(Vector2int16.new(-1, 3))

('aaa')

(Vector2int16.new(1, 3))

(true, false, true)

('aaaa')

('', '', '')

({'', ''})

({true, false})

('a', 'a')

({Vector2int16.new(-1, 3)})

(Vector3int16.new(1, 3, 5))

(Vector3int16.new(-1, 3, -5))

(Vector3int16.new(1, -3, 5))

(Vector3int16.new(-1, -3, -5))

('aaaaa')

({'a', 'a'})

('', '', '', '')

(true, false, true, false)

('aaaaaa')

({true, false, true})

({'', '', ''})

(Vector2.zero)

(0)

(18375)

({Vector3int16.new(-1, 3, -5)})

(Vector2.new(-1, 2.5))

('a', 'a', 'a')

(Vector2.new(-1, -2.73))

('aaaaaaa')

(-18375)

(Vector2.new(1, -2))

(Vector2.one)

(Vector2.new(1, 2))

(Vector2int16.new(-1, 3), Vector2int16.new(-1, 3))

('aaaaaaaa')

({true, false, true, false})

({'', '', '', ''})

('', '', '', '', '')

(true, false, true, false, true)

('aaaaaaaaa')

({0})

({'a', 'a', 'a'})

({Vector2.new(1, 2)})

({Vector2int16.new(-1, 3), Vector2int16.new(-1, 3)})

('', '', '', '', '', '')

({'', '', '', '', ''})

('aaaaaaaaaa')

('a', 'a', 'a', 'a')

(true, false, true, false, true, false)

({true, false, true, false, true})

(Vector3.one)

(Vector3.zero)

(Vector3.new(1, -2, 3))

(Vector3.new(1, 2, 3))

(Vector3.new())

('aaaaaaaaaaa')

(Vector3.new(-1, 2.5, -3.27))

(CFrame.identity)

({true, false, true, false, true, false})

('', '', '', '', '', '', '')

('aaaaaaaaaaaa')

(true, false, true, false, true, false, true)

({'', '', '', '', '', ''})

(CFrame.new(1, -2, 3))

(CFrame.new(-1, 2.5, -3.27))

(Vector3int16.new(-1, 3, -5), Vector3int16.new(-1, 3, -5))

(CFrame.new(1, 2, 3))

({'a', 'a', 'a', 'a'})

(CFrame.new())

('a', 'a', 'a', 'a', 'a')

({Vector3.new(1, 2, 3)})

(Vector2int16.new(-1, 3), Vector2int16.new(-1, 3), Vector2int16.new(-1, 3))

('aaaaaaaaaaaaa')

({'', '', '', '', '', '', ''})

('', '', '', '', '', '', '', '')

(true, false, true, false, true, false, true, false)

('aaaaaaaaaaaaaa')

({true, false, true, false, true, false, true})

({CFrame.identity})

({Vector3int16.new(-1, 3, -5), Vector3int16.new(-1, 3, -5)})

({Vector2int16.new(-1, 3), Vector2int16.new(-1, 3), Vector2int16.new(-1, 3)})

('aaaaaaaaaaaaaaa')

({'a', 'a', 'a', 'a', 'a'})

(Vector2.new(1, 2), Vector2.new(1, 2))

('aaaaaaaaaaaaaaaa')

(0, 0)

('', '', '', '', '', '', '', '', '')

({true, false, true, false, true, false, true, false})

({'', '', '', '', '', '', '', ''})

('a', 'a', 'a', 'a', 'a', 'a')

('aaaaaaaaaaaaaaaaa')

(CFrame.fromEulerAnglesYXZ(1, -2, 3))

({Vector2.new(1, 2), Vector2.new(1, 2)})

({0, 0})

({'', '', '', '', '', '', '', '', ''})

({'a', 'a', 'a', 'a', 'a', 'a'})

(CFrame.fromEulerAnglesYXZ(1, 2, 3) + Vector3.new(1, 2, 3))

(CFrame.fromEulerAnglesYXZ(-1, 2.5, -3.27) + Vector3.new(-1, -2, -3))

(CFrame.fromEulerAnglesYXZ(1, 2, 3))

(CFrame.fromEulerAnglesYXZ(1, -2, 3) + Vector3.new(-1, 2, 3))

(CFrame.fromEulerAnglesYXZ(-1, 2.5, -3.27))

('aaaaaaaaaaaaaaaaaa')

('aaaaaaaaaaaaaaaaaaa')

('a', 'a', 'a', 'a', 'a', 'a', 'a')

(Vector3int16.new(-1, 3, -5), Vector3int16.new(-1, 3, -5), Vector3int16.new(-1, 3, -5))

({CFrame.fromEulerAnglesYXZ(1, 2, 3)})

('aaaaaaaaaaaaaaaaaaaa')

({Vector3int16.new(-1, 3, -5), Vector3int16.new(-1, 3, -5), Vector3int16.new(-1, 3, -5)})

({'a', 'a', 'a', 'a', 'a', 'a', 'a'})

('aaaaaaaaaaaaaaaaaaaaa')

('a', 'a', 'a', 'a', 'a', 'a', 'a', 'a')

('aaaaaaaaaaaaaaaaaaaaaa')

('aaaaaaaaaaaaaaaaaaaaaaa')

(Vector3.new(1, 2, 3), Vector3.new(1, 2, 3))

({'a', 'a', 'a', 'a', 'a', 'a', 'a', 'a'})

('aaaaaaaaaaaaaaaaaaaaaaaa')

(Vector2.new(1, 2), Vector2.new(1, 2), Vector2.new(1, 2))

('aaaaaaaaaaaaaaaaaaaaaaaaa')

(0, 0, 0)

(CFrame.new(), CFrame.new())

('aaaaaaaaaaaaaaaaaaaaaaaaaa')

({Vector3.new(1, 2, 3), Vector3.new(1, 2, 3)})

({0, 0, 0})

('aaaaaaaaaaaaaaaaaaaaaaaaaaa')

({Vector2.new(1, 2), Vector2.new(1, 2), Vector2.new(1, 2)})

('aaaaaaaaaaaaaaaaaaaaaaaaaaaa')

({CFrame.identity, CFrame.identity})

('aaaaaaaaaaaaaaaaaaaaaaaaaaaaa')

('aaaaaaaaaaaaaaaaaaaaaaaaaaaaaa')

('aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa')

('aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa')

('aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa')

('aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa')

(0, 0, 0, 0)

('aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa')

({0, 0, 0, 0})

('aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa')

('aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa')

(Vector3.new(1, 2, 3), Vector3.new(1, 2, 3), Vector3.new(1, 2, 3))

(CFrame.fromEulerAnglesYXZ(1, 2, 3), CFrame.fromEulerAnglesYXZ(1, 2, 3))

(CFrame.fromEulerAnglesYXZ(1, 2, 3) + Vector3.new(1, 2, 3), CFrame.fromEulerAnglesYXZ(1, 2, 3) + Vector3.new(1, 2, 3))

('aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa')

('aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa')

({Vector3.new(1, 2, 3), Vector3.new(1, 2, 3), Vector3.new(1, 2, 3)})

('aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa')

({CFrame.fromEulerAnglesYXZ(1, 2, 3), CFrame.fromEulerAnglesYXZ(1, 2, 3)})

(CFrame.new(), CFrame.new(), CFrame.new())

('aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa')

('aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa')

({CFrame.identity, CFrame.identity, CFrame.identity})

(0, 0, 0, 0, 0)

('aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa')

('aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa')

('aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa')

({0, 0, 0, 0, 0})

('aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa')

('aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa')

('aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa')

(0, 0, 0, 0, 0, 0)

({0, 0, 0, 0, 0, 0})

(CFrame.fromEulerAnglesYXZ(1, 2, 3), CFrame.fromEulerAnglesYXZ(1, 2, 3), CFrame.fromEulerAnglesYXZ(1, 2, 3))

(CFrame.fromEulerAnglesYXZ(1, 2, 3) + Vector3.new(1, 2, 3), CFrame.fromEulerAnglesYXZ(1, 2, 3) + Vector3.new(1, 2, 3), CFrame.fromEulerAnglesYXZ(1, 2, 3) + Vector3.new(1, 2, 3))

({CFrame.fromEulerAnglesYXZ(1, 2, 3), CFrame.fromEulerAnglesYXZ(1, 2, 3), CFrame.fromEulerAnglesYXZ(1, 2, 3)})

(0, 0, 0, 0, 0, 0, 0)

({0, 0, 0, 0, 0, 0, 0})

(0, 0, 0, 0, 0, 0, 0, 0)

({0, 0, 0, 0, 0, 0, 0, 0})

(0, 0, 0, 0, 0, 0, 0, 0, 0)

({0, 0, 0, 0, 0, 0, 0, 0, 0})