Exploring Python Packages — ctypes and cffi
Though Python is really fast in prototyping, it is very slow as compared to C or C++ when a program is executed. You can execute C program in Python by creating a shared library and using ctypes and cffi packages. This gives better speed as compared to the same functionality written purely in Python. Let’s see this in action!
Installation:
A few things to install before we can try this out. First things first, if you have anaconda installed I believe you should have ctypes installed by default (because I did not 😅). You will have to install cffi package by using the command — conda install -c anaconda cffi.
You will also need a C compiler and an editor for writing C code. I used CodeBlocks as editor and mingw as compiler. Be warned though! If you choose bundled mingw installer for CodeBlocks it installs 32-bit compiler by default. You will have to download mingw64 bit compiler which is available at — https://sourceforge.net/projects/mingw-w64/files/. Because I did not know this important part, I was not able to properly execute the code in Python and wasted a lot of time 😥. Also I did some changes in compiler settings to compile for 64 bit options as below:
Furthermore there is another tab ‘ Toolchain executables’ where I changed all the exes path to the mingw64 folder exes (shown in below image):
Now that we have seen the prerequisites let’s see the C code required. This code finds the number of primes between two numbers. This has a while statement and for loop for displaying the primes.
void displayValues(int* startPointer,int* counterVal)
{ int tempVal = *counterVal;
for(int iCount = 0;iCount < tempVal;iCount++)
{ printf("\n%d", *(startPointer + iCount));
}
}void getPrimeNumber(int high,int low,int* counter)
{ int i, flag, tempAdd;
int* arrayStartAddr;
int memNeed = (int)(0.5 * high);
arrayStartAddr = malloc(memNeed * sizeof(int));
if(arrayStartAddr == NULL)
{
printf("Error! memory not allocated.");
exit(0);
}while (low < high)
{ flag = 0;
// ignore numbers less than 2
if (low <= 1)
{ ++low;
continue;
}
for (i = 2; i <= low / 2; ++i)
{
if (low % i == 0)
{
flag = 1;
break;
}
}
if (flag == 0)
{ //printf("\n %d", low);
*(arrayStartAddr + *counter) = low;
tempAdd = *counter + 1;
*counter = tempAdd;
}
++low;
}
displayValues(arrayStartAddr,counter);
free(arrayStartAddr);
}
Now to use this code in Python, we have to make a DLL. The DLL can be created in CodeBlocks by selecting New Project ->Shared Library File. You just need to copy the above code and click on Build. This will generate the DLL of the code.
Python Program:
The below code too calculates primes albeit in python.
import time
def findPrimeNumbers(lower,upper):
allVal = []
for num in range(lower, upper + 1):
# all prime numbers are greater than 1
if num > 1:
for i in range(2, num):
if (num % i) == 0:
break
else:
allVal.append(num)
return allVal
t0 = time.time()
for iCount in range (1,100000):
allVal = findPrimeNumbers(1,1000)
t1 = time.time()
total = t1-t0
print(total)Python Program for ctypes:
To utilize the C code in Python we will use the ctypes. ctypes is a foreign function library for Python. It provides C compatible data types, and allows calling functions in DLLs or Shared Libraries. It can be used to wrap these libraries in pure Python. The below code shows the usage of ctypes to call the C function
from ctypes import *
import time
fun = WinDLL('libprimeNum.dll')
fun.getPrimeNumber.argtypes = [c_int,c_int,POINTER(c_int)]
t0 = time.time()
for iCount in range (1,100000):
fun.getPrimeNumber(c_int(1000),c_int(1),pointer(c_int(0)))
t1 = time.time()
total = t1-t0
print(total)Python Program for cffi:
cffi is an external package providing a C Foreign Function Interface for Python. cffi allows one to interact with almost any C code from Python. You need to add C-like declarations to Python code. cffi has two different main modes, “ ABI” and “ API”. In ABI mode one accesses the library at binary level, while in API mode a separate compilation step with a C compiler is used. ABI mode can be easier to start with, but API mode is faster and more robust and is thus normally the recommended mode. In this we have used the ABI level in line. The code below shows the DLL used.
import time
from cffi import FFI
ffi = FFI()
lib = ffi.dlopen("libprimeNum.dll")
# Describe the data type and function prototype to cffi.
ffi.cdef(''' void getPrimeNumber(int high,int low,int* counter); ''')
t0 = time.time()
for iCount in range (1,100000):
high = ffi.cast("int",1000)
low = ffi.cast("int",1)
counter = ffi.new("int *")
counter[0] = 0
dout = lib.getPrimeNumber(high,low,counter)
t1 = time.time()
total = t1-t0
print(total)The Comparison:
The speed with single run is very fast in all the three cases. But if the function is executed repeatedly in a for loop with large values, we get a lot of difference in time. After executing these codes in Python we can see the time results below:
The file cffiTrial.py uses the cffi module whereas the ctypesTrial.py uses the ctypes module. findPrime.py is the pure Python script.
You can see that the code written in C executes 10 times faster than the Python code. There could definitely be more optimization in the code written. But this exercise shows how fast C is as compared to Python. It may also be possible that not every time this is needed. Only in cases where speed is of utmost importance such type of changes can be included.
With this exercise we can see that by using modules like ctypes and cffi we can use the C code in Python and make the execution faster. Let me know in comments if this example was helpful!!!!
References:
- A lot of googling amongst which the major sources were stackoverflow, programiz, realpython,cffi and ctypes documentation.
Originally published at http://evrythngunder3d.wordpress.com on February 19, 2021.
