• BitManipulation

I'm looking for information on Python bit manipulation, binary manipulation.

It seems that there are no modules for performing Python bit manipulation.

I personally want to be able to:

• Turn "11011000111101..." into bytes, (padded left or right, 0 or 1,) and vice versa.
• Slice ranges of bits
• Rotate bits, addressed by the bit. That is, say: "rotate bits 13-17, wrapping around the edges," or, "rotate bits 13-17, lose bits on the one side, set all new bits to 0."
• Similarly, revert regions of bits, apply logic to regions of bits, etc.,.
• Switch Endianness, with different block sizes.
• Apply operations in block groupings: ex: apply XOR 10101 (5 bits) repeatedly across a field.

The closest thing I've found is ASPN: bit-field manipulation.

I imagine that there are many more manipulations people would like to do with bits.

Manipulations

To integer.

   1 >>> print int('00100001', 2)
   2 33

To hex string. Note that you don't need to use x8 bits.

   1 >>> print "0x%x" % int('11111111', 2)
   2 0xff
   3 >>> print "0x%x" % int('0110110110', 2)
   4 0x1b6
   5 >>> print "0x%x" % int('0010101110101100111010101101010111110101010101', 2)
   6 0xaeb3ab57d55

To character. 8 bits max.

   1 >>> chr(int('111011', 2))
   2 ';'
   3 >>> chr(int('1110110', 2))
   4 'v'
   5 >>> chr(int('11101101', 2))
   6 '\xed'

Characters to integers, but not to strings of 1's and 0's.

   1 >>> int('01110101', 2)
   2 117
   3 >>> chr(int('01110101', 2))
   4 'u'
   5 >>> ord('u')
   6 117

Individual bits.

   1 >>> 1 << 0
   2 1
   3 >>> 1 << 1
   4 2
   5 >>> 1 << 2
   6 4
   7 >>> 1 << 3
   8 8
   9 >>> 1 << 4
  10 16
  11 >>> 1 << 5
  12 32
  13 >>> 1 << 6
  14 64
  15 >>> 1 << 7
  16 128

Transformations Summary

Strings to Integers:

• "1011101101": int(str, 2)

• "m": ord(str)

• "0xdecafbad": int(str, 16) (known to work in Python 2.4)

• "decafbad": int(str, 16) (known to work in Python 2.4)

Integers to Strings:

• "1011101101": built-in to Python 3 (see below)

• "m": chr(str)

• "0xdecafbad": hex(val)

• "decafbad": "%x" % val

We are still left without a technique for producing binary strings, and decyphering hex strings.

Hex String to Integer

The simplest approach is to use the int type with the base argument.

   1 >>> int('0xff',16)
   2 255
   3 >>> int('d484fa894e',16)
   4 912764078414

Another approach to decyphering "0xdecafbad" style hex strings, is to use eval:

   1 >>> eval("0xdecafbad ")
   2 3737844653L

However, this could be dangerous, depending on where you're getting your data from.

Here's a function that is safer:

   1 def hex_to_integer(h):
   2     """Convert a hex string to an integer.
   3 
   4     The hex string can be any length. It can start with an 0x, or not.
   5     Unrecognized characters will raise a ValueError.
   6 
   7     This function released into the public domain by it's author, Lion
   8     Kimbro.
   9     """
  10     num = 0  # Resulting integer
  11     h = h.lower()  # Hex string
  12     if h[:2] == "0x":
  13         h = h[2:]
  14     for c in h:  # Hex character
  15         num = num * 16
  16         if "0" <= c <= "9":
  17             num = num + (ord(c) - ord("0"))
  18         elif "a" <= c <= "f":
  19             num = num + (ord(c) - ord("a"))
  20             num = num + 10
  21         else:
  22             raise ValueError(c)
  23     return num

Integer to Bin String

Python 3 supports binary literals (e.g. 0b10011000) and has a bin() function. For older versions:

   1 >>> def bin(a):
   2         s=''
   3         t={'0':'000','1':'001','2':'010','3':'011',
   4            '4':'100','5':'101','6':'110','7':'111'}
   5         for c in oct(a)[1:]:
   6                 s+=t[c]
   7         return s

Research Links

this is the sort of thing we're looking for:

• ASPN: bit-field manipulation

related modules:

• array module -- (issued with Python)

• struct module -- (issued with Python)

• binascii module -- (issued with Python)

• pySerial module -- access the serial port

see also: BitwiseOperators