bitmath.getsize()¶

bitmath.getsize(path[, bestprefix=True[, system=NIST]])¶

Return a bitmath instance representing the size of a file at any given path.

Parameters:	path (string) – The path of a file to read the size of bestprefix (bool) – Default: True, the returned instance will be in the best human-readable prefix unit. If set to False the result is a bitmath.Byte instance. system (One of bitmath.NIST or bitmath.SI) – Default: bitmath.NIST. The preferred system of units for the returned instance.

Internally bitmath.getsize() calls os.path.realpath() before calling os.path.getsize() on any paths.

Here’s an example of where we’ll run bitmath.getsize() on the bitmath source code using the defaults for bestprefix and system:

>>> import bitmath
>>> print bitmath.getsize('./bitmath/__init__.py')
33.3583984375 KiB

Let’s say we want to see the results in bytes. We can do this by setting bestprefix to False:

>>> import bitmath
>>> print bitmath.getsize('./bitmath/__init__.py', bestprefix=False)
34159.0 Byte

Recall, the default for representation is with the best human-readable prefix. We can control the prefix system used by setting system to either bitmath.NIST (the default) or bitmath.SI:

1 2 3 4 5 6

>>> print bitmath.getsize('./bitmath/__init__.py') 33.3583984375 KiB >>> print bitmath.getsize('./bitmath/__init__.py', system=bitmath.NIST) 33.3583984375 KiB >>> print bitmath.getsize('./bitmath/__init__.py', system=bitmath.SI) 34.159 kB

We can see in lines 1 → 4 that the same result is returned when system is not set and when system is set to bitmath.NIST (the default).

New in version 1.0.7.

bitmath.listdir()¶

bitmath.listdir(search_base[, followlinks=False[, filter='*'[, relpath=False[, bestprefix=False[, system=NIST]]]]])¶

This is a generator which recurses a directory tree yielding 2-tuples of:

• The absolute/relative path to a discovered file
• A bitmath instance representing the apparent size of the file

Parameters:

search_base (string) – The directory to begin walking down followlinks (bool) – Default: False, do not follow links. Whether or not to follow symbolic links to directories. Setting to True enables directory link following filter (string) – Default: * (everything). A glob to filter results with. See fnmatch for more details about globs relpath (bool) – Default: False, returns the fully qualified to each discovered file. True to return the relative path from the present working directory to the discovered file. If relpath is False, then bitmath.listdir() internally calls os.path.realpath() to normalize path references bestprefix (bool) – Default: False, returns bitmath.Byte instances. Set to True to return the best human-readable prefix unit for representation system (One of bitmath.NIST or bitmath.SI) – Default: bitmath.NIST. Set a prefix preferred unit system. Requires bestprefix is True

Note

• This function does not return tuples for directory entities. Including directories in results is scheduled for introduction in the upcoming 1.1.0 release.
• Symlinks to files are followed automatically

When interpreting the results from this function it is crucial to understand exactly which items are being taken into account, what decisions were made to select those items, and how their sizes are measured.

Results from this function may seem invalid when directly compared to the results from common command line utilities, such as du, or tree.

Let’s pretend we have a directory structure like the following:

some_files/
├── deeper_files/
│   └── second_file
└── first_file

Where some_files/ is a directory, and so is some_files/deeper_files/. There are two regular files in this tree:

• somefiles/first_file - 1337 Bytes
• some_files/deeper_files/second_file - 13370 Bytes

The total size of the files in this tree is 1337 + 13370 = 14707 bytes.

Let’s call bitmath.listdir() on the some_files/ directory and see what the results look like. First we’ll use all the default parameters, then we’ll set relpath to True:

1 2 3 4 5 6 7 8 9 10 11

>>> import bitmath >>> for f in bitmath.listdir('./some_files'): ... print f ... ('/tmp/tmp.P5lqtyqwPh/some_files/first_file', Byte(1337.0)) ('/tmp/tmp.P5lqtyqwPh/some_files/deeper_files/second_file', Byte(13370.0)) >>> for f in bitmath.listdir('./some_files', relpath=True): ... print f ... ('some_files/first_file', Byte(1337.0)) ('some_files/deeper_files/second_file', Byte(13370.0))

On lines 5 and 6 the results print the full path, whereas on lines 10 and 11 the path is relative to the present working directory.

Let’s play with the filter parameter now. Let’s say we only want to include results for files whose name begins with “second”:

>>> for f in bitmath.listdir('./some_files', filter='second*'):
...     print f
...
('/tmp/tmp.P5lqtyqwPh/some_files/deeper_files/second_file', Byte(13370.0))

If we wish to avoid having to write for-loops, we can collect the results into a list rather simply:

>>> files = list(bitmath.listdir('./some_files'))
>>> print files
[('/tmp/tmp.P5lqtyqwPh/some_files/first_file', Byte(1337.0)), ('/tmp/tmp.P5lqtyqwPh/some_files/deeper_files/second_file', Byte(13370.0))]

Here’s a more advanced example where we will sum the size of all the returned results and then play around with the possible formatting. Recall that a bitmath instance representing the size of the discovered file is the second item in each returned tuple.

>>> discovered_files = [f[1] for f in bitmath.listdir('./some_files')]
>>> print discovered_files
[Byte(1337.0), Byte(13370.0)]
>>> print reduce(lambda x,y: x+y, discovered_files)
14707.0 Byte
>>> print reduce(lambda x,y: x+y, discovered_files).best_prefix()
14.3623046875 KiB
>>> print reduce(lambda x,y: x+y, discovered_files).best_prefix().format("{value:.3f} {unit}")
14.362 KiB

New in version 1.0.7.

bitmath.parse_string()¶

bitmath.parse_string(str_repr)¶

Parse a string representing a unit into a proper bitmath object. All non-string inputs are rejected and will raise a ValueError. Strings without units are also rejected. See the examples below for additional clarity.

Parameters:	str_repr (string) – The string to parse. May contain whitespace between the value and the unit.
Returns:	A bitmath object representing str_repr
Raises ValueError:
	if str_repr can not be parsed

A simple usage example:

>>> import bitmath
>>> a_dvd = bitmath.parse_string("4.7 GiB")
>>> print type(a_dvd)
<class 'bitmath.GiB'>
>>> print a_dvd
4.7 GiB

Caution

Caution is advised if you are reading values from an unverified external source, such as output from a shell command or a generated file. Many applications (even /usr/bin/ls) still do not produce file size strings with valid (or even correct) prefix units.

To protect your application from unexpected runtime errors it is recommended that calls to bitmath.parse_string() are wrapped in a try statement:

>>> import bitmath
>>> try:
...     a_dvd = bitmath.parse_string("4.7 G")
... except ValueError:
...    print "Error while parsing string into bitmath object"
...
Error while parsing string into bitmath object

Here we can see some more examples of invalid input, as well as two acceptable inputs:

>>> import bitmath
>>> sizes = [ 1337, 1337.7, "1337", "1337.7", "1337 B", "1337B" ]
>>> for size in sizes:
...     try:
...         print "Parsed size into %s" % bitmath.parse_string(size).best_prefix()
...     except ValueError:
...         print "Could not parse input: %s" % size
...
Could not parse input: 1337
Could not parse input: 1337.7
Could not parse input: 1337
Could not parse input: 1337.7
Parsed size into 1.3056640625 KiB
Parsed size into 1.3056640625 KiB

New in version 1.1.0.

bitmath.format()¶

bitmath.format([fmt_str=None[, plural=False[, bestprefix=False]]])¶

The bitmath.format() context manager allows you to specify the string representation of all bitmath instances within a specific block of code.

This is effectively equivalent to applying the format() method to an entire region of code.

Parameters:

fmt_str (str) – a formatting mini-language compat formatting string. See the instances attributes for a list of available items. plural (bool) – True enables printing instances with trailing s‘s if they’re plural. False (default) prints them as singular (no trailing ‘s’) bestprefix (bool) – True enables printing instances in their best human-readable representation. False, the default, prints instances using their current prefix unit.

Note

The bestprefix parameter is not yet implemented!

Let’s look at an example of toggling pluralization on and off. First we’ll look over a demonstration script (below), and then we’ll review the output.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34

import bitmath a_single_bit = bitmath.Bit(1) technically_plural_bytes = bitmath.Byte(0) always_plural_kbs = bitmath.kb(42) formatting_args = { 'not_plural': a_single_bit, 'technically_plural': technically_plural_bytes, 'always_plural': always_plural_kbs } print """None of the following will be pluralized, because that feature is turned off """ test_string = """ One unit of 'Bit': {not_plural} 0 of a unit is typically said pluralized in US English: {technically_plural} several items of a unit will always be pluralized in normal US English speech: {always_plural}""" print test_string.format(**formatting_args) print """ ---------------------------------------------------------------------- """ print """Now, we'll use the bitmath.format() context manager to print the same test string, but with pluralization enabled. """ with bitmath.format(plural=True): print test_string.format(**formatting_args)

The context manager is demonstrated in lines 33 → 34. In these lines we use the bitmath.format() context manager, setting plural to True, to print the original string again. By doing this we have enabled pluralized string representations (where appropriate). Running this script would have the following output:

None of the following will be pluralized, because that feature is turned off

   One unit of 'Bit': 1.0 Bit

   0 of a unit is typically said pluralized in US English: 0.0 Byte

   several items of a unit will always be pluralized in normal US English
   speech: 42.0 kb

----------------------------------------------------------------------

Now, we'll use the bitmath.format() context manager
to print the same test string, but with pluralization enabled.

   One unit of 'Bit': 1.0 Bit

   0 of a unit is typically said pluralized in US English: 0.0 Bytes

   several items of a unit will always be pluralized in normal US English
   speech: 42.0 kbs

Here’s a shorter example, where we’ll:

• Print a string containing bitmath instances using the default formatting (lines 2 → 3)
• Use the context manager to print the instances in scientific notation (lines 4 → 7)
• Print the string one last time to demonstrate how the formatting automatically returns to the default format (lines 8 → 9)

1 2 3 4 5 6 7 8 9

>>> import bitmath >>> print "Some instances: %s, %s" % (bitmath.KiB(1 / 3.0), bitmath.Bit(512)) Some instances: 0.333333333333 KiB, 512.0 Bit >>> with bitmath.format("{value:e}-{unit}"): ... print "Some instances: %s, %s" % (bitmath.KiB(1 / 3.0), bitmath.Bit(512)) ... Some instances: 3.333333e-01-KiB, 5.120000e+02-Bit >>> print "Some instances: %s, %s" % (bitmath.KiB(1 / 3.0), bitmath.Bit(512)) Some instances: 0.333333333333 KiB, 512.0 Bit

New in version 1.0.8.

Class Method: from_other()¶

bitmath class objects have one public class method, BitMathClass.from_other() which provides an alternative way to initialize a bitmath class.

This method may be called on bitmath class objects directly. That is to say: you do not need to call this method on an instance of a bitmath class, however that is a valid use case.

classmethod BitMathClass.from_other(item)¶

Instantiate any BitMathClass using another instance as reference for it’s initial value.

The from_other() class method has one required parameter: an instance of a bitmath class.

Parameters:	item (BitMathInstance) – An instance of a bitmath class.
Returns:	a bitmath instance of type BitMathClass equivalent in value to item
Return type:	BitMathClass
Raises TypeError:
	if item is not a valid bitmath class

In pure Python, this could also be written as:

1
2
3
4
5
6
7
8
9

In [1]: a_mebibyte = MiB(1)

In [2]: a_mebibyte_sized_kibibyte = KiB(bytes=a_mebibyte.bytes)

In [3]: a_mebibyte == a_mebibyte_sized_kibibyte
Out[3]: True

In [4]: print a_mebibyte, a_mebibyte_sized_kibibyte
1.0MiB 1024.0KiB

to_THING()¶

Like the available classes, there are 24 to_THING() methods available. THING is any of the bitmath classes. You can even to_THING() an instance into itself again:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

In [1]: from bitmath import *

In [2]: one_mib = MiB(1)

In [3]: one_mib_in_kb = one_mib.to_kb()

In [4]: one_mib == one_mib_in_kb

Out[4]: True

In [5]: another_mib = one_mib.to_MiB()

In [6]: print one_mib, one_mib_in_kb, another_mib

1.0 MiB 8388.608 kb 1.0 MiB

In [7]: six_TB = TB(6)

In [8]: six_TB_in_bits = six_TB.to_Bit()

In [9]: print six_TB, six_TB_in_bits

6.0 TB 4.8e+13 Bit

In [10]: six_TB == six_TB_in_bits

Out[10]: True

best_prefix()¶

best_prefix([system=None])¶

Return an equivalent instance which uses the best human-readable prefix-unit to represent it.

Parameters:	system (int) – one of bitmath.NIST or bitmath.SI
Returns:	An equivalent bitmath instance
Return type:	bitmath
Raises ValueError:
	if an invalid unit system is given for system

The best_prefix() method returns the result of converting a bitmath instance into an equivalent instance using a prefix unit that better represents the original value. Another way to think of this is automatic discovery of the most sane, or human readable, unit to represent a given size. This functionality is especially important in the domain of interactive applications which need to report file sizes or transfer rates to users.

As an analog, consider you have 923,874,434¢ in your bank account. You probably wouldn’t want to read your bank statement and find your balance in pennies. Most likely, your bank statement would read a balance of $9,238,744.34. In this example, the input prefix is the cent: ¢. The best prefix for this is the dollar: $.

Let’s, for example, say we are reporting a transfer rate in an interactive application. It’s important to present this information in an easily consumable format. The library we’re using to calculate the rate of transfer reports the rate in bytes per second from a tx_rate() function.

We’ll use this example twice. In the first occurrence, we will print out the transfer rate in a more easily digestible format than pure bytes/second. In the second occurrence we’ll take it a step further, and use the format method to make the output even easier to read.

In [9]: for _rate in tx_rate():
    print "Rate: %s/second" % Bit(_rate)
    time.sleep(1)

Rate: 100.0 Bit/sec
Rate: 24000.0 Bit/sec
Rate: 1024.0 Bit/sec
Rate: 60151.0 Bit/sec
Rate: 33.0 Bit/sec
Rate: 9999.0 Bit/sec
Rate: 9238742.0 Bit/sec
Rate: 2.09895849555e+13 Bit/sec
Rate: 934098021.0 Bit/sec
Rate: 934894.0 Bit/sec

And now using a custom formatting definition:

In [50]: for _rate in tx_rate():
    print Bit(_rate).best_prefix().format("Rate: {value:.3f} {unit}/sec")
    time.sleep(1)

Rate: 12.500 Byte/sec
Rate: 2.930 KiB/sec
Rate: 128.000 Byte/sec
Rate: 7.343 KiB/sec
Rate: 4.125 Byte/sec
Rate: 1.221 KiB/sec
Rate: 1.101 MiB/sec
Rate: 2.386 TiB/sec
Rate: 111.353 MiB/sec
Rate: 114.123 KiB/sec

format()¶

BitMathInstance.format(fmt_spec)¶

Return a custom-formatted string to represent this instance.

Parameters:	fmt_spec (str) – A valid formatting mini-language string
Returns:	The custom formatted representation
Return type:	string

bitmath instances come with a verbose built-in string representation:

In [1]: leet_bits = Bit(1337)

In [2]: print leet_bits
1337.0 Bit

However, for instances which aren’t whole numbers (as in MiB(1/3.0) == 0.333333333333 MiB, etc), their representation can be undesirable.

The format() method gives you complete control over the instance’s representation. All of the instances attributes are available to use when choosing a representation.

The following sections describe some common use cases of the format() method as well as provide a brief tutorial of the greater Python formatting meta-language.

In [1]: ugly_number = MB(50).to_MiB() / 8.0
In [2]: print ugly_number
5.96046447754 MiB

In [3]: print ugly_number.format("{value:.2f}{unit}")
5.96 MiB

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22

In [8]: longer_format = """Formatting attributes for %s
   ...: This instances prefix unit is {unit}, which is a {system} type unit
   ...: The unit value is {value}
   ...: This value can be truncated to just 1 digit of precision: {value:.1f}
   ...: In binary this looks like: {binary}
   ...: The prefix unit is derived from a base of {base}
   ...: Which is raised to the power {power}
   ...: There are {bytes} bytes in this instance
   ...: The instance is {bits} bits large
   ...: bytes/bits without trailing decimals: {bytes:.0f}/{bits:.0f}""" % str(ugly_number)

In [9]: print ugly_number.format(longer_format)
Formatting attributes for 5.96046447754 MiB
This instances prefix unit is MiB, which is a NIST type unit
The unit value is 5.96046447754
This value can be truncated to just 1 digit of precision: 6.0
In binary this looks like: 0b10111110101111000010000000
The prefix unit is derived from a base of 2
Which is raised to the power 20
There are 6250000.0 bytes in this instance
The instance is 50000000.0 bits large
bytes/bits without trailing decimals: 6250000/50000000

THING Properties¶

Like the available classes, there are 24 THING properties available. THING is any of the bitmath classes. Under the covers these properties call to_THING.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21

In [1]: from bitmath import *

In [2]: one_mib = MiB(1)

In [3]: one_mib == one_mib.kb

Out[3]: True

In [4]: print one_mib, one_mib.kb, one_mib.MiB

1.0 MiB 8388.608 kb 1.0 MiB

In [5]: six_TB = TB(6)

In [6]: print six_TB, six_TB.Bit

6.0 TB 4.8e+13 Bit

In [7]: six_TB == six_TB.Bit

Out[7]: True

Arithmetic¶

Math works mostly like you expect it to, except for a few edge-cases:

• Mixing bitmath types with Number types (the result varies per-operation)
• Operations where two bitmath types would cancel out (such as dividing two bitmath types)
• Multiplying two bitmath instances together is supported, but the results may not make much sense.

Bitwise Operations¶

Bitwise operations are also supported. Bitwise operations work directly on the bits attribute of a bitmath instance, not the number you see in an instances printed representation (value), to maintain accuracy.

1. Give me a break here, it’s not easy coming up with compelling examples for bitwise operations...

The Hard Way¶

Core Networking Values

• net.core.rmem_max - Bytes - Single Value - Default receive buffer size
• net.core.wmem_max - Bytes - Single Value - Default write buffer size

System-Wide Memory Limits

• net.ipv4.tcp_mem - Pages - Three Value Vector - The max field of the parameter is the number of memory pages allowed for queueing by all TCP sockets.

Per-Socket Buffers

Per-socket buffer sizes must not exceed the core networking buffer sizes.

• net.ipv4.tcp_rmem - Bytes - Three Field Vector - The max field sets the size of the TCP receive buffer
• net.ipv4.tcp_wmem - Bytes - Three Field Vector - As above, but for the write buffer

We would normally calculate the optimal BDP and related values following this approach:

1. Measure the latency, or round trip time (RTT, measured in milliseconds), between the host we’re tuning and our target remote host
2. Measure/identify our network transfer rate
3. Calculate the BDP (multiply transfer rate by rtt)
4. Obtain our current kernel settings
5. Adjust settings as necessary

But for the sake brevity we’ll be working out of an example scenario with a pre-defined RTT and transfer rate.

Scenario

• We have an average network transfer rate of 1Gb/sec (where Gb is the SI unit for Gigabits, not Gibibytes: GiB)
• Our latency (RTT) is 0.199ms (milliseconds)

Calculate Manually

Lets calculate the BDP now. Because the kernel parameters expect values in units of bytes and pages we’ll have to convert our transfer rate of 1Gb/sec into B/s (Gigabits/second to Bytes/second):

• Convert 1Gb into an equivalent byte based unit

Remember 1 Byte = 8 Bits:

tx_rate_GB = 1/8 = 0.125

Our equivalent transfer rate is 0.125GB/sec.

• Convert our RTT from milliseconds into seconds

Remember 1ms = 10^-3s:

window_seconds = 0.199 * 10^-3 = 0.000199

Our equivalent RTT window is 0.000199s

• Next we multiply the transfer rate by the length of our RTT window (in seconds)

(The unit analysis for this is GB/s * s leaving us with GB)

BDP = rx_rate_GB * window_seconds = 0.125 * 0.000199 = 0.000024875

Our BDP is 0.000024875GB.

• Convert 0.000024875GB to bytes:

Remember 1GB = 10⁹B

BDP_bytes = 0.000024875 * 10^9 = 24875.0

Our BDP is 24875 bytes (or about 24.3KiB)

The bitmath way¶

All of this math can be done much quicker (and with greater accuracy) using the bitmath library. Let’s see how:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

>>> from bitmath import GB

>>> tx = 1/8.0

>>> rtt = 0.199 * 10**-3

>>> bdp = (GB(tx * rtt)).to_Byte()

>>> print bdp.to_KiB()

KiB(24.2919921875)

We could shorten that even further:

>>> print (GB((1/8.0) * (0.199 * 10**-3))).to_Byte()
24875.0Byte

Get the current kernel parameters

Important to note is that the per-socket buffer sizes must not exceed the core network buffer sizes. Lets fetch our current core buffer sizes:

$ sysctl net.core.rmem_max net.core.wmem_max
net.core.rmem_max = 212992
net.core.wmem_max = 212992

Recall, these values are in bytes. What are they in KiB?

>>> print Byte(212992).to_KiB()
KiB(208.0)

This means our core networking buffer sizes are set to 208KiB each. Now let’s check our current per-socket buffer sizes:

$ sysctl net.ipv4.tcp_rmem net.ipv4.tcp_wmem
net.ipv4.tcp_rmem = 4096        87380   6291456
net.ipv4.tcp_wmem = 4096        16384   4194304

Let’s double-check that our buffer sizes aren’t already out of wack (per-socket should be <= networking core)

>>> net_core_max = KiB(bytes=212992)

>>> ipv4_tcp_rmem_max = KiB(bytes=6291456)

>>> ipv4_tcp_rmem_max > net_core_max

True

It appears that my buffers aren’t sized appropriately. We’ll fix that when we set the tunable parameters.

Finally, how large is the entire system TCP buffer?

$ sysctl net.ipv4.tcp_mem
net.ipv4.tcp_mem = 280632       374176  561264

Our max system TCP buffer size is set to 561264. Recall that this parameter is measured in memory pages. Most of the time your page size is 4096 bytes, but you can check by running the command: getconf PAGESIZE. To convert the system TCP buffer size (561264) into a byte-based unit, we’ll multiply it by our pagesize (4096):

>>> sys_pages = 561264

>>> page_size = 4096

>>> sys_buffer = Byte(sys_pages * page_size)

>>> print sys_buffer.to_MiB()

2192.4375MiB

>>> print sys_buffer.to_GiB()

2.14105224609GiB

The system max TCP buffer size is about 2.14GiB.

In review, we discovered the following:

• Our core network buffer size is insufficient (212992), we’ll set it higher
• Our current per-socket buffer sizes are 6291456 and 4194304

And we calculated the following:

• Our ideal max per-socket buffer size is 24875 bytes
• Our ideal default per-socket buffer size (half the max): 12437

Finally: Set the new kernel parameters

Set the core-network buffer sizes:

$ sudo sysctl net.core.rmem_max=24875  net.core.wmem_max=24875
net.core.rmem_max = 4235
net.core.wmem_max = 4235

Set the per-socket buffer sizes:

$ sudo sysctl net.ipv4.tcp_rmem="4096 12437 24875" net.ipv4.tcp_wmem="4096 12437 24875"
net.ipv4.tcp_rmem = 4096 12437 24875
net.ipv4.tcp_wmem = 4096 12437 24875

And it’s done! Testing this is left as an exercise for the reader. Note that in my experience this is less useful on wireless connections.

Terminology¶

The definitions describes some of the terminology used throughout this section.

Coercion

The act of converting operands into a common type to support arithmetic operations. Somewhat similar to how adding two fractions requires coercing each operand into having a common denominator.

Specific to the bitmath domain, this concept means using an instance’s prefix value for mixed-math.

Operand

The object(s) of a mathematical operation. That is to say, given 1 + 2, the operands would be 1 and 2.

Operator

The mathematical operation to evaluate. Given 1 + 2, the operation would be addition, +.

LHS

Left-hand side. In discussion this specifically refers to the operand on the left-hand side of the operator.

RHS

Right-hand side. In discussion this specifically refers to the operand on the right-hand side of the operator.

Two bitmath operands¶

This section describes what happens when two bitmath instances are used as operands. There are three possible results from this type of operation.

Addition and subtraction

The result will be of the type of the LHS.

Multiplication

Supported, but yields strange results.

1
2
3
4
5
6
7
8
9

In [10]: first = MiB(5)

In [11]: second = kB(2)

In [12]: first * second
Out[12]: MiB(10000.0)

In [13]: (first * second).best_prefix()
Out[13]: GiB(9.765625)

As we can see on lines 6 and 9, multiplying even two relatively small quantities together (MiB(5) and kB(2)) yields quite large results.

Internally, this is implemented as:

Division

The result will be a number type due to unit cancellation.

Mixed Types: Addition and Subtraction¶

This describes the behavior of addition and subtraction operations where one operand is a bitmath type and the other is a number type.

Mixed-math addition and subtraction always return a type from the numbers family (integer, float, long, etc...). This rule is true regardless of the placement of the operands, with respect to the operator.

Discussion: Why do 100 - KiB(90) and KiB(100) - 90 both yield a result of 10.0 and not another bitmath instance, such as KiB(10.0)?

When implementing the math part of the object datamodel customizations[2] there were two choices available:

1. Offer no support at all. Instead raise a NotImplemented exception.
2. Consistently apply coercion to the bitmath operand to produce a useful result (useful if you know the rules of the library).

In the end it became a philosophical decision guided by scientific precedence.

Put simply, bitmath uses the significance of the least significant operand, specifically the number-type operand because it lacks semantic significance. In application this means that we drop the semantic significance of the bitmath operand. That is to say, given an input like GiB(13.37) (equivalent to == 13.37 * 2³⁰), the only property used in calculations is the prefix value, 13.37.

Numbers carry mathematical significance, in the form of precision, but what they lack is semantic (contextual) significance. A number by itself is just a measurement of an arbitrary quantity of stuff. In mixed-type math, bitmath effectively treats numbers as mathematical constants.

A bitmath instance also has mathematical significance in that an instance is a measurement of a quantity (bits in this case) and that quantity has a measurable precision. But a bitmath instance is more than just a measurement, it is a specialized representation of a count of bits. This gives bitmath instances semantic significance as well.

And so, in deciding how to handle mixed-type (really what we should say is mixed-significance) operations, we chose to model the behavior off of an already established set of rules. Those rules are the Rules of Significance Arithmetic[3].

Let’s look at an example of this in action:

In [8]: num = 42

In [9]: bm = PiB(24)

In [10]: print num + bm
66.0

Equivalently, divorcing the bitmath instance from it’s value (this is coercion):

In [12]: bm_value = bm.value

In [13]: print num + bm_value
66.0

What it all boils down to is this: if we don’t provide a unit then bitmath won’t give us one back. There is no way for bitmath to guess what unit the operand was intended to carry. Therefore, the behavior of bitmath is conservative. It will meet us half way and do the math, but it will not return a unit in the result.

Mixed Types: Multiplication and Division¶

Multiplication has commutative properties. This means that the ordering of the operands is not significant. Because of this fact bitmath allows arbitrary placement of the operands, treating the numeric operand as a constant. Here’s an example demonstrating this.

In [2]: 10 * KiB(43)
Out[2]: KiB(430.0)

In [3]: KiB(43) * 10
Out[3]: KiB(430.0)

Division, however, does not have this commutative property. I.e., the placement of the operands is significant. Additionally, there is a semantic difference in division. Dividing a quantity (e.g. MiB(100)) by a constant (10) makes complete sense. Conceptually (in the domain of bitmath), the intention of MiB(100) / 10) is to separate MiB(10) into 10 equal sized parts.

In [4]: KiB(43) / 10
Out[4]: KiB(4.2998046875)

The reverse operation does not maintain semantic validity. Stated differently, it does not make logical sense to divide a constant by a measured quantity of stuff. If you’re still not clear on this, ask yourself what you would expect to get if you did this:

Footnotes¶

Changes¶

Added Functionality

• New bitmath command-line tool added. Provides CLI access to basic unit conversion functions
• New utility function bitmath.parse_string for parsing a human-readable string into a bitmath object. Patch submitted by new contributor tonycpsu.

Major Updates¶

• bitmath has a proper documentation website up now on Read the Docs, check it out: bitmath.readthedocs.org
• bitmath is now Python 3.x compatible
• bitmath is now included in the Extra Packages for Enterprise Linux EPEL6 and EPEL7 repositories
• merged 6 pull requests from 3 contributors

Bug Fixes¶

• fixed some math implementation bugs
  • commutative multiplication
  • true division

Changes¶

Added Functionality

• best-prefix guessing: automatic best human-readable unit selection
• support for bitwise operations
• formatting customization methods (including plural/singular selection)
• exposed many more instance attributes (all instance attributes are usable in custom formatting)
• a context manager for applying formatting to an entire block of code
• utility functions for sizing files and directories
• add instance properties equivalent to instance.to_THING() methods

Project¶

Tests

• Test suite is now implemented using Python virtualenv’s for consistency across across platforms
• Test suite now contains 150 unit tests. This is 110 more tests than the previous major release (1.0.4-1)
• Test suite now runs on EPEL6 and EPEL7
• Code coverage is stable around 95-100%

Project¶

Available via:

• PyPi
• Fedora 19
• Fedora 20

bitmath had been under development for 12 days when the 1.0.4-1 release was made available.

Debut Functionality¶

• Converting between SI and NIST prefix units (GiB to kB)
• Converting between units of the same type (SI to SI, or NIST to NIST)
• Basic arithmetic operations (subtracting 42KiB from 50GiB)
• Rich comparison operations (1024 Bytes == 1KiB)
• Sorting
• Useful console and print representations