Python Static Code Analysis

Python static code analysis with Prospector and Bandit

Not going to talk about the "why should one use code analysis tools", if you read it up to the end you'll get it!

Which code analysis tools?

Let's talk about Code Analysis in Python, what to use, why to use, experience on using it.

Currently I use only 2, and for me it's the limit of code analysis tools to use at the same time without much overhead.  Prospector & Bandit. Why two?

Well one, Bandit,  focus specifically on security on your Python code, the other has a wider range focusing on potential errors, complexity and convention violations . Both overlap in some way, with Prospector replacing Pylint, pep8 and McCabe complexity, endoing: you'll get more than enough code analysis, sometime more than you can handle. (I'll get to that latter on)

 

How to Install Bandit and Prospector?

Installation is very strait forward on both of them.

Installing Bandit is as simple as:

pip install bandit

or 

pip3 install bandit 

Installing Prospector isn't much harder:

pip install prospector[with_everything]

or 

pip3 install prospector[with_everything]

 

Running of Bandit and Prospector?

First you need your python file. In this case I've used an example one just for the sake of example, let's call it tst.py

It's a very simple program with one function, user input and screen output. Let's run first Bandit, how? Simple:

    bandit tst.py

Here's the result:

Looks like we've got a potential issue with the "input" , since we're using python3 won't be a problem but lets assume that it is. Go to the link and read the explanation, so if we're on Python2 let's swap input() with raw_input() else it's all alright.

Now running prospector, the -s flag points the profiles used, from veryhigh to verylow, in here you can opt to choose, if you don't use the flag medium shall be used, for sake of example let's go for the veryhigh :

    prospector -s veryhigh tst.py

 

 

As you can see LOTS of warnings are issued in the veryhigh (30 vs 6 on the medium profile) so use the veryhigh and high with come caution because most of the warnings in veryhigh are related to syntax and indentation aspects (pep8).

So if you apply most of the changes/fix all the warnings how will the code look like? ( DO keep in mind that sometimes you can't actually apply all the changes due to legacy code/ code dependencies, and sometimes your editor my leave some spaces/tabs between the lines which will conflict with the "veryhigh" settings)

Now your code is a bit safer and cleaner. This was a simple and silly example, try to run it on your own scripts/programs and see the differences, again in prospector don't rush into the "veryhigh" profile.

 

Speed up apt-get updates

How to speed up apt-get updates?

Apt is the package manager used on Ubuntu and Debian systems, it's been around for a while and it should be around for a lot more, yet it has a problem: it downloads only one package at a time thus making it slow sometimes, specially if you have tons of tinny packages to update.

The solution: apt-fast! Downloads packages in parallel making A LOT faster.

• How to install

sudo add-apt-repository ppa:apt-fast/stable
sudo apt-get update
sudo apt-get -y install apt-fast

• How to use it

sudo apt-fast update
sudo apt-fast upgrade

apt-fast installation

That's it!  

Obviously the more packages you're installing/upgrading the more you'll notice the difference.

Lenovo X250 tweeking in linux

Why the Lenovo X250?

My needs: a daily driver laptop, very cheap, light, small, upgradable and serviceable. (want to swap hdd, thermal paste etc) and Linux friendly. All things considered, I came up with a bargain on eBay, a 12.5"  Lenovo ThinkPad X250, i5 5300U with 8gb RAM, 128gb SSD, 2 batteries and HD screen with a barely noticeable bruise (which shall be swapped latter) for 130Euro.

X250 condition

X250 keyboard and screen

The Lenovo X250 in 2019 It's preparation for linux daily driver.

The batteries, yep no typo --two batteries-- this model has 2, one internal the other external were ~82% capacity each, the screen bruise is somehow noticeable, the fan and thermals were alright, yet first things to do; swap thermal paste for a top of the line one and swap the 128gb SSD for one bigger a 256SSD.  Keep in mind that there's a whitelist of LCD screens, if not on the list no brightness control on windows.

So price tag till now:

• X250 + Postage = 130Euro
• Grizzly  Kryonaut = 5Euro
• Western Digital Green SSD 256Gb = 33Euro
• FHD IPS screen ref ( ref MTM 20CLS0XA03 20CL)  = 75Euro

• Linux Mint19 = Free

X250 Interior

The thermal paste change settled the temperatures in full load (15min test) under 65 degrees, ( s-tui tool to monitor and stress CPU, sysbench to stress).

The i5-5300U CPU has a 'built in' GPU, the HD5500 and together both can consume up to 15w of power. When this power consumption threshold is exceeded the frequencies are lowered and the performance of the CPU, the GPU or both it takes a hit. The CPU/GPU throttling can also happen when the one of both achieves a certain temperature threshold, in the case of the X250 this doesn't happen due to good cooling. Even with default cooling/paste I could run stress tests at full CPU speed.

Undervolting Lenovo X250 and optimizing battery life

Next I decided to undervolt the CPU, this would allow even lower temperatures, more battery life and avoid package throttling, the 15watt limit! If both CPU and GPU consume less, less throttling will happen (duhh) and more performance one will get.
To undervolt I used undervolt python package, after a couple tests I could lower my voltages with the following values:

--core -100 --cache -100 --gpu -55 --uncore -70 --analogio -50

The values might get lower but I want full stability under all circumstances.

Hint: Don't just do stress tests, use the computer normally, and use it while doing the tests, I happen to pass lots of tests with -130mV in the core but crashed when opening firefox for example.

In order to test I installed sysbench and ran the following command:

sudo sysbench cpu --threads=4 --time=300 run

My results were the following, yet what matters is that the temperature didn't go above 63 degress Celcius.

CPU speed:
    events per second:  2675.87

General statistics:
    total time:                          300.0013s
    total number of events:              802769

Latency (ms):
         min:                                  1.09
         avg:                                  1.49
         max:                                 30.35
         95th percentile:                      1.52
         sum:                            1199489.59

Threads fairness:
    events (avg/stddev):           200692.2500/865.84
    execution time (avg/stddev):   299.8724/0.00

5 minutes after the test the temperatures drop to an very acceptable 37 degrees!

Battery life: depends a lot on what I do, compiling, watching youtube, browsing, screen brightness, but safe to say that at 50% screen brightness, light browsing 5 hours easily.

Now 2 important packages needed to be installed, these (basically) will tuneup the SO so it consumes a lot less energy, TLP will even make your fan be quieter in normal use: powertop and tlp

To install TLP:

$ sudo add-apt-repository ppa:linrunner/tlp
$ sudo apt-get update
$ sudo apt-get install tlp tlp-rdw 

$ sudo tlp start 

To install and configure powertop:

$ sudo apt-get update
$ sudo apt-get install powertop

$ sudo powertop --auto-tune 

On my lenovo X250 these steps gave me an extra 1h battery life and LOT quieter fan.