1. The `bitmath` Module

1.1. Functions 

This section describes utility functions included in the bitmath module.

1.1.2. bitmath.listdir()

bitmath.listdir(search_base[, followlinks=False[, glob='*'[, relpath=False[, bestprefix=False[, system=NIST]]]]])[source]

This is a generator which recurses a directory tree yielding 2-tuples of:

The absolute/relative path to a discovered file
A bitmath instance representing the apparent size of the file

Parameters:

search_base (str or pathlib.Path) – The directory to begin walking down. May be a plain string or a pathlib.Path object.
followlinks (bool) – Default: False, do not follow links. Whether or not to follow symbolic links to directories. Setting to True enables directory link following
glob (string) – Default: * (everything). A glob to filter results with. See fnmatch for more details about globs
relpath (bool) – Default: False, returns the fully qualified to each discovered file. True to return the relative path from the present working directory to the discovered file. If relpath is False, then bitmath.listdir() internally calls os.path.realpath() to normalize path references
bestprefix (bool) – Default: False, returns Byte instances. Set to True to return the best human-readable prefix unit for representation
system (One of bitmath.NIST or bitmath.SI) – Default: bitmath.NIST. Set a prefix preferred unit system. Requires bestprefix is True

Note

Symlinks to files are followed automatically

Deprecated since version 2.1.0: bitmath.listdir() is deprecated and will be removed in a future release. Use os.walk() with bitmath.getsize() directly.

Given this directory tree:

some_files/
├── deeper_files/
│   └── second_file   (13370 bytes)
└── first_file        (1337 bytes)

>>> import bitmath

>>> # Default: absolute paths, Byte instances
>>> for path, size in bitmath.listdir('./some_files'):
...     print(path, size)
/tmp/some_files/first_file Byte(1337.0)
/tmp/some_files/deeper_files/second_file Byte(13370.0)

>>> # Relative paths
>>> for path, size in bitmath.listdir('./some_files', relpath=True):
...     print(path, size)
some_files/first_file Byte(1337.0)
some_files/deeper_files/second_file Byte(13370.0)

>>> # Filter by glob
>>> for path, size in bitmath.listdir('./some_files', glob='second*'):
...     print(path, size)
/tmp/some_files/deeper_files/second_file Byte(13370.0)

>>> # Sum all results
>>> sizes = [size for _, size in bitmath.listdir('./some_files')]
>>> print(sum(sizes).best_prefix())
14.3623046875 KiB

Added in version 1.0.7.

Changed in version 2.0.0: The filter parameter was renamed to glob. Passing filter still works but emits a DeprecationWarning.

1.1.3. bitmath.sum()

bitmath.sum(iterable[, start=None])[source]

Sum an iterable of bitmath instances into a single bitmath instance.

Parameters:

iterable – Any iterable of bitmath objects to sum.
start (A bitmath instance, or None) – Default: None (accumulates into bitmath.Byte). Pass a bitmath instance to set both the starting value and the result type.

Returns:

A bitmath instance whose type is determined by start (or bitmath.Byte when start is None).

Note

Python’s built-in sum() also works with bitmath objects. Because 0 + bm returns bm itself, the built-in accumulates into the type of the first element in the iterable. Use bitmath.sum() instead when you need the result normalised to a specific unit regardless of the input types.

Sum a homogeneous list — result type matches start (Byte by default):

>>> import bitmath
>>> bitmath.sum([bitmath.MiB(1), bitmath.GiB(1)])
Byte(1074790400.0)

Pass start to choose a different accumulator unit:

>>> bitmath.sum([bitmath.KiB(1), bitmath.KiB(2)], start=bitmath.MiB(0))
MiB(0.0029296875)

Contrast with the built-in sum(), whose result type tracks the first element:

>>> sum([bitmath.KiB(1), bitmath.KiB(2)])
KiB(3.0)
>>> sum([bitmath.Byte(1), bitmath.MiB(1), bitmath.GiB(1)])
Byte(1074790401.0)

1.1.4. bitmath.parse_string()

bitmath.parse_string(str_repr, system=bitmath.NIST, strict=True)[source]

Parse a string (or, when strict=False, a string or number) into a bitmath object.

Parameters:

str_repr – The value to parse. String inputs may include whitespace between the value and the unit.
system – Unit system used when strict=False and the intended unit cannot be reliably determined from the input. Ignored when strict=True. One of bitmath.NIST (default) or bitmath.SI.
strict – When True (default) the unit must be an exact bitmath type name such as "KiB" or "MB". When False the parser accepts ambiguous input such as plain numbers, numeric strings, and case-insensitive single-letter units. See parse_string with strict=False below.

Returns:

A bitmath object representing the input.

Raises:

ValueError – if the input cannot be parsed.

A simple usage example:

>>> import bitmath
>>> a_dvd = bitmath.parse_string("4.7 GiB")
>>> print(type(a_dvd))
<class 'bitmath.GiB'>
>>> print(a_dvd)
4.7 GiB

Caution

Caution is advised when reading values from an unverified external source such as shell command output or a generated file. Many applications (even /usr/bin/ls) do not produce file size strings with valid prefix units unless specially configured. Use strict=False for those cases — see parse_string with strict=False.

To protect your application from unexpected runtime errors, wrap calls in a try statement:

>>> import bitmath
>>> try:
...     a_dvd = bitmath.parse_string("4.7 G")
... except ValueError:
...    print("Error while parsing string into bitmath object")
...
Error while parsing string into bitmath object

Here are some more examples of valid and invalid input:

>>> import bitmath
>>> sizes = [ 1337, 1337.7, "1337", "1337.7", "1337 B", "1337B" ]
>>> for size in sizes:
...     try:
...         print("Parsed size into %s" % bitmath.parse_string(size).best_prefix())
...     except ValueError:
...         print("Could not parse input: %s" % size)
...
Could not parse input: 1337
Could not parse input: 1337.7
Could not parse input: 1337
Could not parse input: 1337.7
Parsed size into 1.3056640625 KiB
Parsed size into 1.3056640625 KiB

Changed in version 1.2.4: Added support for parsing octet units via issue #53 - parse french units. The usage of “octet” is still common in some RFCs, as well as France, French Canada and Romania. See also, a table of the octet units and their values on Wikipedia.

Here are some simple examples of parsing octet based units:

>>> import bitmath
>>> a_mebibyte = bitmath.parse_string("1 MiB")
>>> a_mebioctet = bitmath.parse_string("1 Mio")
>>> print(a_mebibyte, a_mebioctet)
1.0 MiB 1.0 MiB
>>> print(bitmath.parse_string("1Po"))
1.0 PB
>>> print(bitmath.parse_string("1337 Eio"))
1337.0 EiB

Notice how on lines 4 and 5 the variable a_mebibyte from the input "1 MiB" is exactly equivalent to a_mebioctet from "1 Mio". After parsing, octet units are normalised into their standard NIST/SI equivalents automatically.

Changed in version 2.0.0: Added strict and system parameters. The default strict=True behavior is identical to earlier versions. system defaults to bitmath.NIST and is only consulted when strict=False.

Added in version 1.1.0.

1.1.4.1. parse_string with `strict=False`

When strict=False the parser accepts ambiguous input that does not conform to exact bitmath type names — for example, the single-letter units produced by tools like ls -h, df, and qemu-img. This is the behavior previously provided by the now-deprecated bitmath.parse_string_unsafe().

All inputs are treated as byte-based. Bit-based units are not supported in non-strict parsing mode. Capitalisation does not matter.

1.1.4.1.1. Understanding the `system` parameter in non-strict mode

Important

In strict=False mode, system is a tiebreaker, not a guarantee. It is only consulted when the parser cannot determine the unit system from the input itself. Passing system=bitmath.SI does not force all results to be SI units.

The parser resolves the unit system in the following order of precedence:

No unit present — plain numbers and numeric strings (e.g. 100, "2048") are always returned as bitmath.Byte regardless of system.
Unit is self-describing — inputs whose unit already contains an i marker (e.g. "100 KiB", "4Gi") unambiguously identify a NIST unit. system is ignored and the result is always NIST.
Unit is ambiguous — single-letter units such as k, M, G carry no inherent system information. Only here does system act as the deciding hint: system=bitmath.NIST (the default) interprets "4G" as GiB(4); passing system=bitmath.SI interprets it as GB(4).

In summary: system resolves ambiguity — it does not override evidence already present in the input string.

In this example we parse the output of df -H / /boot /home, whose Used column contains single-letter SI units. Because the units are ambiguous we pass system=bitmath.SI as a hint:

Filesystem                                 Size  Used Avail Use% Mounted on
/dev/mapper/luks-ca8d5493-72bb-4691-afe1   107G   64G   38G  63% /
/dev/sda1                                  500M  391M   78M  84% /boot
/dev/mapper/vg_deepfryer-lv_home           129G  118G  4.7G  97% /home

>>> with open('/tmp/df-output.txt', 'r') as fp:
...     _ = fp.readline()   # skip header
...     for line in fp.readlines():
...         cols = line.split()[0:4]
...         print("""Filesystem: %s
... - Used: %s""" % (cols[0],
...             bitmath.parse_string(cols[1], strict=False, system=bitmath.SI)))
Filesystem: /dev/mapper/luks-ca8d5493-72bb-4691-afe1
- Used: 107.0 GB
Filesystem: /dev/sda1
- Used: 500.0 MB
Filesystem: /dev/mapper/vg_deepfryer-lv_home
- Used: 129.0 GB

If df is run with -h instead of -H it produces NIST-sized values but still prints the same single-letter units. Omit system (NIST is the default) or pass system=bitmath.NIST explicitly:

>>> with open('/tmp/df-output.txt', 'r') as fp:
...     _ = fp.readline()   # skip header
...     for line in fp.readlines():
...         cols = line.split()[0:4]
...         print("""Filesystem: %s
... - Used: %s""" % (cols[0],
...             bitmath.parse_string(cols[1], strict=False, system=bitmath.NIST)))
Filesystem: /dev/mapper/luks-ca8d5493-72bb-4691-afe1
- Used: 100.0 GiB
Filesystem: /dev/sda1
- Used: 477.0 MiB
Filesystem: /dev/mapper/vg_deepfryer-lv_home
- Used: 120.0 GiB

The results now use the proper NIST prefix syntax: GiB.

1.1.5. bitmath.parse_string_unsafe()

Deprecated since version 2.0.0: parse_string_unsafe is deprecated and will be removed in a future release. Use bitmath.parse_string() with strict=False instead:

# old
bitmath.parse_string_unsafe(value, system=bitmath.NIST)

# new
bitmath.parse_string(value, strict=False, system=bitmath.NIST)

To suppress the deprecation warning in the interim:

import warnings
warnings.filterwarnings('ignore', category=DeprecationWarning,
                        module='bitmath')

bitmath.parse_string_unsafe(repr[, system=bitmath.NIST])[source]

A deprecated thin wrapper around parse_string(repr, strict=False, system=system). All behavior, parameters, and caveats are identical to parse_string with strict=False.

Parameters:

repr – The value to parse.
system – bitmath.NIST (default) or bitmath.SI.

Returns:

A bitmath object representing repr.

Raises:

ValueError – if repr cannot be parsed.

Changed in version 2.0.0: Deprecated. Default system changed from bitmath.SI to bitmath.NIST for consistency with bitmath.parse_string().

Added in version 1.3.1.

1.1.6. bitmath.query_capacity()

bitmath.query_capacity(path, bestprefix=True, system=NIST)[source]

Return the total, used, and free capacity of the volume containing path as a Capacity NamedTuple of bitmath.Bitmath instances. This is the recommended API for volume size queries — it works cross-platform without elevated privileges.

Parameters:

path – A path on the volume to query (str or os.PathLike). On Windows, a bare drive letter such as "C" or "C:" is automatically normalized to "C:\\".
bestprefix (bool) – When True (default), each returned field is pre-normalized via best_prefix() for human-readable output. When False, raw bitmath.Byte instances are returned.
system (int) – Prefix system used when bestprefix is True. bitmath.NIST (default) gives binary prefixes (GiB); bitmath.SI gives decimal prefixes (GB). Ignored when bestprefix is False.

Returns:

A bitmath.Capacity with fields total, used, and free.

Raises:

FileNotFoundError – if path does not exist.
PermissionError – if the process lacks access to query path.

Linux examples (human-readable by default):

>>> import bitmath
>>> cap = bitmath.query_capacity("/")
>>> print(cap.total)
465.762 GiB
>>> cap = bitmath.query_capacity("/home")
>>> print(cap.free)
120.5 GiB

macOS examples:

>>> cap = bitmath.query_capacity("/")
>>> print(cap.total)
500.068 GiB
>>> cap = bitmath.query_capacity("/Volumes/SomeVolume")
>>> print(cap.used)
14.311 GiB

Windows examples (bare drive letters are normalized automatically):

>>> cap = bitmath.query_capacity("C:\\")
>>> print(cap.total)
476.837 GiB
>>> cap = bitmath.query_capacity("C")
>>> print(cap.free)
200.0 GiB

Tuple unpacking:

>>> total, used, free = bitmath.query_capacity("/")
>>> print(total, used, free)
465.762 GiB 186.264 GiB 279.397 GiB

Opt into SI (decimal) prefixes:

>>> cap = bitmath.query_capacity("/", system=bitmath.SI)
>>> print(cap.total)
500.068 GB

Raw bitmath.Byte instances (no prefix normalization):

>>> cap = bitmath.query_capacity("/", bestprefix=False)
>>> print(cap.total)
500068036608.0 Byte

Added in version 2.0.0.

1.1.7. bitmath.query_device_capacity()

bitmath.query_device_capacity(device_fd)[source]

Query the raw physical capacity of a block device. This is an advanced, privileged API intended for callers that need the true hardware capacity (e.g. disk imaging tools). Most users should use query_capacity() instead.

Requires root on Linux and administrator on Windows. Not supported on macOS — SIP blocks raw block device access even for root; use query_capacity() there.

Parameters:

device_fd (file) – An open file handle of the block device.

Returns:

A bitmath.Byte equal to the device capacity.

Raises:

ValueError – if device_fd is not a block device.
NotImplementedError – on macOS or any unsupported platform.
OSError – if the underlying ioctl or DeviceIoControl call fails.

Linux example (requires root):

>>> import bitmath
>>> with open("/dev/sda", "rb") as device:
...     size = bitmath.query_device_capacity(device).best_prefix()
...     print(f"Device {device.name} capacity: {size}")
Device /dev/sda capacity: 238.475 GiB

Windows example (requires administrator privileges):

>>> import bitmath
>>> drives = [r'\\.\PhysicalDrive0', r'\\.\PhysicalDrive1']
>>> for path in drives:
...     with open(path, 'rb') as drive:
...         size = bitmath.query_device_capacity(drive).best_prefix()
...         print(f"Drive {path}: {size}")
Drive \\.\PhysicalDrive0: 80.0 GiB
Drive \\.\PhysicalDrive1: 14.311 TiB

Note

macOS: this function raises NotImplementedError because System Integrity Protection (SIP) prevents raw block device access. Use query_capacity() instead.

Added in version 1.2.4.

Changed in version 2.0.0: Added Windows support. macOS now raises NotImplementedError (SIP restriction).

1.2. Context Managers 

This section describes all of the context managers provided by the bitmath class.

Note

Context manager settings are thread safe. Concurrent contexts in different threads do not interfere with each other.

Note

For a bit of background, a context manager (specifically, the with statement) is a feature of the Python language which is commonly used to:

Decorate, or wrap, an arbitrary block of code. I.e., effect a certain condition onto a specific body of code
Automatically open and close an object which is used in a specific context. I.e., handle set-up and tear-down of objects in the place they are used.

1.2.1. bitmath.format()

bitmath.format([fmt_str=None[, plural=False[, bestprefix=False]]])[source]

The bitmath.format() context manager allows you to specify the string representation of all bitmath instances within a specific block of code.

This is effectively equivalent to applying the format() method to an entire region of code.

Parameters:

fmt_str (str) – a formatting mini-language compat formatting string. See the instance attributes for a list of available items.
plural (bool) – True enables printing instances with trailing s’s if they’re plural. False (default) prints them as singular (no trailing ‘s’)
bestprefix (bool) – True enables printing instances in their best human-readable representation. False, the default, prints instances using their current prefix unit.

Let’s look at an example of toggling pluralization on and off. First we’ll look over a demonstration script (below), and then we’ll review the output.

import bitmath

a_single_bit = bitmath.Bit(1)
technically_plural_bytes = bitmath.Byte(0)
always_plural_kbs = bitmath.kb(42)

formatting_args = {
    'not_plural': a_single_bit,
    'technically_plural': technically_plural_bytes,
    'always_plural': always_plural_kbs
}

print("""None of the following will be pluralized, because that feature is turned off
""")

test_string = """   One unit of 'Bit': {not_plural}

   0 of a unit is typically said pluralized in US English: {technically_plural}

   several items of a unit will always be pluralized in normal US English
   speech: {always_plural}"""

print(test_string.format(**formatting_args))

print("""
----------------------------------------------------------------------
""")

print("""Now, we'll use the bitmath.format() context manager
to print the same test string, but with pluralization enabled.
""")

with bitmath.format(plural=True):
    print(test_string.format(**formatting_args))

The context manager is demonstrated in lines 33 → 34. In these lines we use the bitmath.format() context manager, setting plural to True, to print the original string again. By doing this we have enabled pluralized string representations (where appropriate). Running this script would have the following output:

None of the following will be pluralized, because that feature is turned off

   One unit of 'Bit': 1.0 b

   0 of a unit is typically said pluralized in US English: 0.0 B

   several items of a unit will always be pluralized in normal US English
   speech: 42.0 kb

----------------------------------------------------------------------

Now, we'll use the bitmath.format() context manager
to print the same test string, but with pluralization enabled.

   One unit of 'Bit': 1.0 b

   0 of a unit is typically said pluralized in US English: 0.0 B

   several items of a unit will always be pluralized in normal US English
   speech: 42.0 kbs

Here’s a shorter example, where we’ll:

Print a string containing bitmath instances using the default formatting (lines 2 → 3)
Use the context manager to print the instances in scientific notation (lines 4 → 7)
Print the string one last time to demonstrate how the formatting automatically returns to the default format (lines 8 → 9)

>>> import bitmath
>>> print("Some instances: %s, %s" % (bitmath.KiB(1 / 3.0), bitmath.Bit(512)))
Some instances: 0.3333333333333333 KiB, 512.0 b
>>> with bitmath.format("{value:e}-{unit}"):
...     print("Some instances: %s, %s" % (bitmath.KiB(1 / 3.0), bitmath.Bit(512)))
...
Some instances: 3.333333e-01-KiB, 5.120000e+02-b
>>> print("Some instances: %s, %s" % (bitmath.KiB(1 / 3.0), bitmath.Bit(512)))
Some instances: 0.3333333333333333 KiB, 512.0 b

Added in version 1.0.8.

1.3. Module Variables 

This section describes the module-level variables. Some of which are constants and are used for reference. Some of which effect output or behavior.

Changed in version 1.0.7: The formatting strings were not available for manupulate/inspection in earlier versions

Added in version 1.1.1: Prior to this version ALL_UNIT_TYPES was not defined

Note

Modifying these variables will change the default representation indefinitely. Use the bitmath.format() context manager to limit changes to a specific block of code.

bitmath.format_string

This is the default string representation of all bitmath instances. The default value is {value} {unit} which, when evaluated, formats an instance as a floating point number with at least one digit of precision, followed by a character of whitespace, followed by the prefix unit of the instance.

For example, given bitmath instances representing the following values: 1337 MiB, 0.1234567 kb, and 0 B, their printed output would look like the following:

>>> from bitmath import *
>>> print(MiB(1337), kb(0.1234567), Byte(0))
1337.0 MiB 0.1234567 kb 0.0 B

We can make these instances print however we want to. Let’s wrap each one in square brackets ([, ]), replace the separating space character with a hyphen (-), and limit the precision to just 2 digits:

>>> import bitmath
>>> bitmath.format_string = "[{value:.2f}-{unit}]"
>>> print(bitmath.MiB(1337), bitmath.kb(0.1234567), bitmath.Byte(0))
[1337.00-MiB] [0.12-kb] [0.00-Byte]

bitmath.format_plural

A boolean which controls the pluralization of instances in string representation. The default is False.

If we wanted to enable pluralization we could set the format_plural variable to True. First, let’s look at some output using the default singular formatting.

>>> import bitmath
>>> print(bitmath.MiB(1337))
1337.0 MiB

And now we’ll enable pluralization (line 2):

>>> import bitmath
>>> bitmath.format_plural = True
>>> print(bitmath.MiB(1337))
1337.0 MiBs
>>> bitmath.format_plural = False
>>> print(bitmath.MiB(1337))
1337.0 MiB

On line 5 we disable pluralization again and then see that the output has no trailing “s” character.

bitmath.NIST: Constant used as an argument to some functions to specify the NIST system.

bitmath.SI: Constant used as an argument to some functions to specify the SI system.

bitmath.SI_PREFIXES: An array of all of the SI unit prefixes (e.g., k, M, or E)

bitmath.SI_STEPS

SI_STEPS = {
    'Bit': 1 / 8.0,
    'Byte': 1,
    'k': 1000,
    'M': 1000000,
    'G': 1000000000,
    'T': 1000000000000,
    'P': 1000000000000000,
    'E': 1000000000000000000,
    'Z': 1000000000000000000000,
    'Y': 1000000000000000000000000
}

bitmath.NIST_PREFIXES: An array of all of the NIST unit prefixes (e.g., Ki, Mi, or Ei)

bitmath.NIST_STEPS

NIST_STEPS = {
    'Bit': 1 / 8.0,
    'Byte': 1,
    'Ki': 1024,
    'Mi': 1048576,
    'Gi': 1073741824,
    'Ti': 1099511627776,
    'Pi': 1125899906842624,
    'Ei': 1152921504606846976,
    'Zi': 1180591620717411303424,
    'Yi': 1208925819614629174706176
}

bitmath.ALL_UNIT_TYPES

An array of all combinations of known valid prefix units mixed with both bit and byte suffixes.

ALL_UNIT_TYPES = ['Bit', 'Byte', 'kb', 'kB', 'Mb', 'MB', 'Gb', 'GB',
   'Tb', 'TB', 'Pb', 'PB', 'Eb', 'EB', 'Zb', 'ZB', 'Yb', 'YB',
   'Kib', 'KiB', 'Mib', 'MiB', 'Gib', 'GiB', 'Tib', 'TiB',
   'Pib', 'PiB', 'Eib', 'EiB', 'Zib', 'ZiB', 'Yib', 'YiB']

1.4. 3rd Party Module Integrations 

Self-contained, copy-paste examples for integrating bitmath with argparse, click, and progressbar2 are collected in the Integration Examples chapter.

1. The bitmath Module

1.1.4.1.1. Understanding the system parameter in non-strict mode

1. The `bitmath` Module

1.1.4.1.1. Understanding the `system` parameter in non-strict mode