You can already use Google Pay in Malta!

Unless you have been living under a rock, you’ll know that Apple Pay is now supported in Malta, through Revolut. But little did Android users know that Google Pay has been supported by Revolut for more than a year now!

According to an article on Android Police, around mid-July 2018, Revolut rolled out support for Google Pay. This means that using any NFC enabled Android device, you can start paying locally with your phone!

What is Revolut?

Revolut is an online bank based in the UK – you can get a free debit card to be used locally and comes with a great Android app. It’s the only card that works with Google Pay in Malta – BOV, HSBC, BNF and any other local bank do not support Google Pay, so Revolut is your only choice in this case. You can sign up with Revolut for free here (note that with each successful sign-up, I get a small kick-back, as described here).

How to I get Google Pay?

Easy – using your Android Phone, just search for Google Pay on Google Play: https://play.google.com/store/apps/details?id=com.google.android.apps.walletnfcrel

Can’t find the app on the store, or it says something like “This app is incompatible with all of your devices”? Just download the APK from some APK Mirror, like https://apkpure.com/google-pay-pay-with-your-phone-and-send-cash/com.google.android.apps.walletnfcrel. Of course, side-loading applications can be risky so proceed with caution.

revolutgooglepay

How do I add my card?

Adding the card is very easy – just follow the setup on screen. If you ever used your Revolut card with Google, you’ll find it right there. If not, just add your details. Have your Card No, Expiry Date and CVV at hand though. You’ll need to confirm a code by SMS as well. No screenshots for this section as Google Pay doesn’t allow you to take screenshots when adding cards. You can find a video guide here – https://www.youtube.com/watch?v=7zS6aR22QZM

Anything I should know?

Yes – keep in mind that these transactions are considered Online transactions. Therefore, you’ll need to enable Online Transactions in the Card Security section. Don’t worry, if it’s turned off by default, Revolut will warn you about this.

onlinetransactionsrevolut

How will my transactions appear?

This is the nice part – they will show up like all other transactions. The payment will not be towards Google; it will point to the retailer you’ve just purchased from.

revoluttransaction

How can I use Google Pay to pay?

Very easy! Firstly, make sure that NFC is turned on. Then, just unlock your phone and tap the payment machine when the retailer asks you to do so. Your phone will vibrate when you get tour phone close enough and Google Pay will activate itself. You may be asked to verify your identity, such as by supplying a PIN, Fingerprint or lock pattern. By, the way, lock patterns are bad and you should avoid them.

Take that, Apple Pay!

C# Micro Optimizations Part 2 – In Parameter Modifier

In this series of posts, we’re investigating micro-optimizations in C#. As previously mentioned, these may not be applicable to all; but it’s still fun looking at these concepts.

Let’s visit back the last post – Ref arguments. Ref arguments gave us the power of passing structs by value in an extremely efficient manner.

Mutability of a ref struct

Passing structs by ref brings a major disadvantage – the callee might mutate the value of the struct without the caller ever knowing. What if we need to pass structs in an efficient manner, whilst having peace of mind that the callee doesn’t mutate the struct?

Meet the in parameter modifier- C# 7.2

What does the in parameter modifier do? It allows us to pass the argument by reference and giving us the guarantee that the arguments cannot be modified by the callee. Excellent! Let’s run a quick test and make sure our performance is still comparable when passing by ref. Let’s have a struct with 2 properties – let’s have some work done using two different methods – passing by ref and passing by in.

All code can be viewed here – https://github.com/albertherd/csharpmopt2-in

public class SixteenBitStructBenchmark
{
    [Benchmark]
    [Arguments(100000000)]
    public void BenchmarkIncrementByRef(int limit)
    {
        SixteenBitStruct sixteenBitStruct = new SixteenBitStruct();
        int counter = 0;
        do
        {
            IncrementByRef(ref sixteenBitStruct);
            counter++;
        }
        while (limit != counter);
    }

    [Benchmark]
    [Arguments(100000000)]
    public void BenchmarkIncrementByIn(int limit)
    {
        SixteenBitStruct sixteenBitStruct = new SixteenBitStruct();
        int counter = 0;
        do
        {
            IncrementIn(sixteenBitStruct);
            counter++;
        }
        while (limit != counter);
    }

    private void IncrementByRef(ref SixteenBitStruct sixteenBitStruct)
    {
        double sum = sixteenBitStruct.D1 + sixteenBitStruct.D2;
    }

    private void IncrementIn(in SixteenBitStruct sixteenBitStruct)
    {
        double sum2 = sixteenBitStruct.D1 + sixteenBitStruct.D2;
    }
}

public struct SixteenBitStruct
{
    public double D1 { get; }
    public double D2 { get; }
}

Let’s see how they perform.

Method limit Mean Error StdDev
BenchmarkIncrementByRef 100000000 23.83 ms 0.0272 ms 0.0241 ms
BenchmarkIncrementByIn 100000000 238.21 ms 0.3108 ms 0.2755 ms

Performance loss?

Wait a second – why is IncrementByIn 10x slower than IncrementByRef when we’re accessing 2 properties in the same struct? Let’s have a look at the generated IL.

IncrementByRef

 IL_0000: ldarg.1
IL_0001: call instance float64 InOperator.SixteenBitStruct::get_D1()
# Loads argument 1 (SixteenBitStruct) and call the getter

IncrementByIn

 IL_0000: ldarg.1
# Prepare a new local variable on the evaluation stack
IL_0001: ldobj InOperator.SixteenBitStruct
# Copies the value of SixteenBitStruct into the loaed argument variable
IL_0006: stloc.0
IL_0007: ldloca.s V_0
IL_0009: call instance float64 InOperator.SixteenBitStruct::get_D1()
# Pops the newly created argument into location 0, loads local variable 0 (new copy of SixteenBitStruct) and call the getter

Interesting! When we’ve called the method by ref, the resultant IL just loads the argument and calls the getter. When we’ve called the method by in, the resultant IL creates a copy of the struct before the getter is called. It seems that each time we’re referencing the property, C# is generating a copy of the object for us? We’re facing a by-design feature – a defensive copy.

Why do we encounter a defensive copy?

When calling the getter of our properties, the compiler doesn’t know if the getter mutates the object. Although this is a getter, it’s only by convention that changes aren’t made; there is no language construct that prevents us from changing values in our getter. The compiler must honor the in keyword and generate a defensive copy, just in case the getter modifies the struct.

In the end of the day, a getter is just syntactic sugar for a method. Of course, defensive copies will be generated if methods are called on the struct since the compiles can’t provide any guarantee that the method call won’t mutate the struct.

How do we get around this?

We’ll need instruct the compiler that our struct is immutable, so the compiler doesn’t need to worry about creating defensive copies since values cannot change. C# provides this exact functionality in fact! We can slap the “readonly” keyword (and drop any setters) so that we can guarantee that our struct is now immutable.

Here’s how it looks now

 public readonly struct SixteenBitStruct
{
    public double D1 { get; }
    public double D2 { get; }
}

Revisiting our performance numbers

Let’s re-run our benchmarks and assess the performance.

Method limit Mean Error StdDev
BenchmarkIncrementByRef 100000000 23.93 ms 0.1226 ms 0.1147 ms
BenchmarkIncrementByIn 100000000 24.06 ms 0.2183 ms 0.2042 ms

Far better! Performance is now equal (within margin of error). Some closing thoughts about this:

  • Using the in operator is an excellent feature – it allows the callers to safely assume that the values they are going to pass will not have their values changed.
  • Using the readonly modifier with a struct is another excellent feature – it allows the the developer to safely say that its value is immutable and no changes are allowed.
  • The performance uplift is should be considered as a bonus – the design and infrastructure wins using the in / readonly keywords in these context carry far more value.
  • Don’t ever use the in keyword in conjunction with non-readonly structs. Chances are that the performance gained from passing by ref will be lost by accessing the struct’s properties and methods.

Until the next one!

C# Micro Optimizations Part 1 – Ref Arguments

In this series of posts, we’ll be investigating key areas for micro-optimizations. As the title implies, these are micro-optimizations and may not be applicable for you unless you are writing some high-performance library of have a piece of code running in a tight loop. Nonetheless, it’s still fun to investigate and find these micro-optimizations. Onwards!

Let’s start with a simple one – the ref keyword in method arguments. For this argument, we’re only concerned with value type method arguments – structs.

Since structs are value types, by default, the entire struct is copied over to the callee, irrelevant of the size of the struct. If the struct is big, this is typically a bottleneck since a copy must be created and passed for each call. C# provides a method of overriding this behavior by using the ref keyword. If an argument is marked as ref, a pointer to the struct will be passed rather than an actual copy!

This brings two major advantages:

  • If the struct is bigger than 4 bytes (on a 32 bit machine) or 8 bytes (on a 64 bit machine), passing a struct by ref means that less data copying is taking place.
  • We avoid copying back the data – we do not need to return the data since a reference is passed rather than a copy of the struct.

Let’s see an example – lets consider a struct containing two doubles – a 16 byte struct. Let’s say we have two methods that increments one of the values for us (just to give the loop something to do and not get it optimised away).

One of them accepts a (copy of a) struct, increments its internal values and returns the copy back. This is passed by value, which is the default behavior for a struct.

The other method accepts a struct by ref and increments its internal values. There is no need to return the data back therefore no extra copies were needed. This is not the default behavior, so we’ll need to accompany it with the ref keyword.

The below is the source code in question – find the whole solution here: https://github.com/albertherd/csharpmopt1-ref

[CoreJob]
public class SixteenBytesStructBenchmark
{
    [Benchmark]
    [Arguments(1000000)]
    public void BenchmarkIncrementByRef(int limit)
    {
        SixteenBytesStruct value = new SixteenBytesStruct();
        int counter = 0;
        do
        {
            IncrementByRef(ref value);
            counter++;
        }
        while (limit != counter);
    }
    [Benchmark]
    [Arguments(1000000)]
    public void BenchmarkIncrementByVal(int limit)
    {
        SixteenBytesStruct value = new SixteenBytesStruct();
        int counter = 0;
        do
        {
            value = IncrementByVal(value);
            counter++;
        }
        while (limit != counter);
    }
    private void IncrementByRef(ref SixteenBytesStruct toIncrement)
    {
        toIncrement.d0++;
    }
    private SixteenBytesStruct IncrementByVal(SixteenBytesStruct toIncrement)
    {
        toIncrement.d0++;
        return toIncrement;
    }
}
public struct SixteenBytesStruct
{
    public long d0, d1;
}

The below is the time taken for 1000000 runs – this was executed using .NET core 2.2.1 – benchmarks done using BenchmarkDotNet

Method limit Mean Error StdDev
BenchmarkIncrementByRef 1000000 1.663 ms 0.0139 ms 0.0130 ms
BenchmarkIncrementByVal 1000000 2.872 ms 0.0155 ms 0.0145 ms

We can see that running this in a tight loop, doing the work by ref, in this case, is 72% faster! To what can we attribute this performance change? Let’s have a look at what’s happening behind the scenes.

Doing the work by value

Calling IncrementByVal

IL_000a: ldarg.0 # Load the “this” parameter on evaluation stack (implicit)
IL_000b: ldloc.0 # Load SixteenBytesStruct value on the stack (16 bytes worth of data) from location 0
IL_000c: call instance valuetype Ref.SixteenBytesStruct Ref.SixteenBytesStructBenchmark::IncrementByVal(valuetype Ref.SixteenBytesStruct) # Call IncrementByVal with the loaded arguments
IL_0011: stloc.0 #Captures the returned value and stores it in location 0

IncrementByVal Implementation

IL_0000: ldarga.s toIncrement # Load the argument’s address so processing can begin
..method work – removed for brevity
IL_001a: ldarg.1 # Load the value of the field back so it can be returned
IL_001b: ret

What’s happening here?

  • Push the value of SixteenBytesStruct ready to be captured by the upcoming method call
  • Call IncrementByVal
  • IncrementByVal loads the address of the received value from the caller and does the required work
  • Push the value of the SixteenByteStruct after the work has been done ready to be captured by the caller
  • IncrementByVal Returns
  • Pop the value from replace the value of SixteenBytesStruct with the new one

Doing the work by ref

Calling IncrementByRef

IL_000a: ldarg.0 # Load the “this” parameter on evaluation stack (implicit)
IL_000b: ldloca.s V_0 # Load SixteenBytesStruct’s address on the stack (8 bytes worth of data)
IL_000d: call instance void Ref.SixteenBytesStructBenchmark::IncrementByRef(valuetype Ref.SixteenBytesStructamp;) # Call IncrementByVal with the loaded arguments

IncrementByVal Implementation

IL_0000: ldarg.1 # Load the argument so processing can begin. We’re not calling ldarga.s since this already the struct’s address rather than the actual value
..method work – removed for brevity
IL_0018: ret # Return

What’s happening here?

  • Push the address of SixteenBytesStruct ready to be captured by the upcoming method call
  • Call IncrementByVal
  • IncrementByVal gets value received from the caller (the value is an address) and does the required work
  • IncrementByVal Returns

What does this mean?

One can obviously note that doing the work by ref has significantly less work to do:

  • The callee is pushing 8 bytes instead of 16 bytes
  • The callee loads 8 bytes onto the evaluation stack instead of 16 bytes
  • The callee doesn’t need to push the new value onto the evaluation stack
  • The callee doesn’t need to pop the stack and stored the updated value

Therefore, doing the work by ref is pushing less data when a method call takes place (maximum of 8 bytes, irrespective of the struct size) and is avoiding two data copy instructions, since it does not need to push and pop the new value since there are no return values.

If you increase the size of the struct, the performance gains would be even bigger, as shown in the below graph.

C#RefBenchmark

We can observe some useful information from this graph

  • When it comes to doing operations by ref, performance is basically equivalent all cross the board, irrelevant to the size of the struct.
  • 16 byte, 8 byte and 4 byte structs carry identical performance – they are just separated by the margin of error.
  • 16 byte, 8 byte and 4 byte structs are faster than 2 byte and 1 byte structs. In fact, 1 byte struct ends up clearly slower than a 2 byte struct! It’s very interesting to explore why 1 and 2 byte structs exhibit performance degradation.
  • The rest of the result show a consistent upward trend – which reflect the amount of data copying take place.

What’s very interesting is that a 4 byte integer operates faster by value when compared to 1 byte and 2 byte integers!

Tag your photo album using Microsoft Cognitive Services

Just interested in the source code? – https://github.com/albertherd/PhotoTagging

Computer vision is one of the key areas that has seen huge growth in both capability and popularity. Though it seems that it’s still out of reach to many; I honestly felt lost when I was trying to play around in this field. It feels like we’re trying to solve problems which have already been solved by other companies. It seems Microsoft shares this vision though, as they’ve introduced Machine Learning features in the form of SaaS.

I’ve stumbled upon Microsoft Cognitive Services through a presentation and I was genuinely amazed. What’s amazing isn’t the results that this service yields – I’ve expected nothing less than excellent results from such tools. What amazed me is how EASY to get involved – there is no fiddling with following pages and pages of guides just to download, install and play around with some software.

Microsoft Cognitive Services enables you to do a huge array of Machine-Learning powered applications, ranging from vision, decision making, natural language processing and other areas. Let’s play around vision – can we use Microsoft’s Cognitive Vision Services and help us organise our photo library?

The idea is that I have many photos, with subjects ranging from food, vacation, family, friends and whatnot. What if my photos contain the proper EXIF tags such as subject and tags? This will allow me to classify my photos by subject and allow me to search through them. What if I can find my photos instantly instead of sifting manually through thousands of photos? I’ll presume that it’s not just me though, everyone has a smartphone nowadays, so this is everyone’s pain.

Great – now we have an objective! Let’s make the tools work for us now. The process will be simple – upload a photo to Microsoft’s Cognitive Vision Services, get the tags and a nice description and slap it to the actual file. Oh, when I say EXIF tags, these can be viewed in File Explorer like below. (Windows 10 Dark Theme in File Explorer here)

ExifInWindowsExplorer

Ready to tag your photo library? Let’s go!

Get a Microsoft Cognitive Services Account

Since this is an online service, you’ll need to have an active account with Microsoft Azure. Get your free account from here. Don’t worry, the free service is more than enough to get you playing around. I’ve used the free tier to develop, test and write this blog and I still have plenty of free capacity left.

Create a new Cognitive Services Resource and get the API key

Now that you have an active Azure account, navigate to the Azure Portal and create a new Cognitive Services Resource. Follow the wizard and get the service created – choose whatever region works best for you. I’ve chosen West Europe and the free tier in my case. Once it’s created, we’ll need two things – the URL to our endpoint and our API Key. From the quick start page, get API endpoint and the API Keys.

Get your photos tagged!

Okay, we got all the resources needed, it’s time to get some work done! I’ve created an application to get a photo, upload it our new Cognitive Services resource, get tags and description and apply it to our photo.
Follow these steps to get your photo tagging game going!

  1. Download / Clone my application from GitHub
  2. Open the application and navigate to PhotoAnalyser.cs. Change the subscriptionKey and uriBase to the ones you got previously. The keys in the solution are placeholder keys only.
  3. Run the application – have your photo directory ready as this is asked for at runtime.
  4. Let it do its magic!

In the below example, photo analysis tells us that it’s a pizza on a plate and it also gave us some appropriate tags. Try downloading and viewing the pizza photo -tags and title are preserved as EXIF data.

Keep in mind that the code in the provided solution is not production ready – it’s merely meant as a playground.

Explore the solution

What’s the fun of having a piece of software working without knowing how it works underlying? Here are some points about the application, in no particular order:

  1. It’s making one of the excellent TPL Dataflow framework from Microsoft – this enables the application to scale with ease and to work around the pesky throttling that the free tier carries with it.
  2. It is resizing the images since they don’t need to be large, plus this speed the process up.
  3. It’s using the ImageSharp to resize and add Exif tags to the images.
  4. Given that this application is manipulating images, it is memory intensive. I’ve seen this image hit close to 4GB in memory usage.
  5. It’s split into a library and a consumer just in case.

Continue exploring the Microsoft Cognitive Services stack

Computer vision is just one of the areas in the Microsoft Cognitive Services stack; there are other excellent services to enrich your applications. They also have excellent documentation on this; I’ve followed this to build my application.

That’s it for today! This was an extremely fun project to learn and experiment with new technologies! Until the next one.

Outputting DS18B20 temperatures on a LCD1602 – Raspberry Pi Temperature Monitoring Part 4

This is part of a tutorial series. If you feel a bit lost, I suggest following the tutorial in order:

Just interested in the code? – https://github.com/albertherd/DS18B20Reader

This article assumes that you’ve configured one or more DS18B20 sensors to your Raspberry Pi and configured your Raspberry Pi to work with an LCD1602. If you did not, read the above mentioned links.

Okay – this should be a quick post – most of the heavy lifting is done. Remember the LCD1602 library that we’ve used in the second tutorial? We’ll be using that to simply get the temperature we’ve captured in third tutorial and display it.

We’ll need to do the following changes:

  1. Import the LCD1602 library.
  2. Initialize the LCD1602 on application startup.
  3. Read and display information on the LCD1602.
  4. Cleanup resources before exiting – this is important since we’ll need to turn off the backlight after usage.

Import the LCD1602 library

Get a copy of this repository – either by cloning or simply copying lcd1602.c and lcd1602.h files to your solution. We’ll be also adding them to our CMake file – it should look something of the sort. I’ve left the solution on debug in this case.

set(SOURCE main.c sensor.c lcd1602.c main.h sensor.h lcd1602.h)
set(CMAKE_BUILD_TYPE Debug)
add_executable(DS18B20Reader ${SOURCE})

Now it’s simply just adding a reference to the lcd1602.h in the solution. Next!

Initialize the LCD1602 on application startup

We’ll need to initialize the library and open a connection to the display under the right address – check your device address by checking the third tutorial. To simplify things, I hard-coded the value which is 0x27 in my case. Call this when initializing the application.

void InitializeLCD()
{
    int rc;
	rc = lcd1602Init(1, LCDADDRESS);
	if (rc)
	{
		printf("Initialization failed; aborting...\n");
		return;
	}
}

Read and display information on the LCD1602

We’ll be modifying our main loop to output content to the console (not important though and output to the LCD1602 screen. The main adjustment we’ve did in the main loop is that we’ve broken down our output to two functions – Outputting to console (not important) and outputting to the LCD1602 – Let’s see the main loop:

void ReadTemperatureLoop(SensorList *sensorList)
{
    while(!sigintFlag)
    {
        for(int i = 0; i SensorCount; i++)
        {
            float temperature = ReadTemperature(sensorList->Sensors[i]);
            PrintTemperatueToLCD1602(sensorList->Sensors[i], i % LCD1602LINES, temperature);
            LogTemperature(sensorList->Sensors[i], temperature);
        }       
    }
}

Let’s now have a look at the important method – PrintTemperatueToLCD1602. Keeping in mind that the LCD1602 has two lines, we’ll be receiving the calculated line number as a parameter. This will make sure that values will lie between 0 and 1 only using modulus.

We’ll also need to remember that each line will hold up to 16 characters, so we’re truncating anything more than 16 characters (actually 16 + 1 for null termination). We’ll then just pass the (potentially truncated) string to the LCD1602 and et voila!

void PrintTemperatueToLCD1602(Sensor *sensor, int lineToPrintDataOn, float temperature)
{
    char temperatureString[LCD1602CHARACTERS + 1];
    snprintf(temperatureString, LCD1602CHARACTERS + 1, "%s : %.2fC", sensor->SensorName, temperature);

    lcd1602SetCursor(0, lineToPrintDataOn);
    lcd1602WriteString(temperatureString);
}

Cleanup resources before exiting

After we’re done, it’s just a matter of cleaning up resources. As previously mentioned, this is important since we’ll need to turn off the backlight after usage. In the cleanup method, we’re just calling the lcd1602Shutdown method.

void Cleanup(SensorList *sensorList)
{
    printf("Exiting...\n");
    FreeSensors(sensorList);
    lcd1602Shutdown();
}

Fetch a complete copy of the code from from GitHub – https://github.com/albertherd/DS18B20Reader

Let’s run the application! In a terminal with git and cmake installed, run the following commands

git clone https://github.com/albertherd/DS18B20Reader
cd ./DS18B20Reader
cmake . && make && ./DS18B20Reader "Sensor"

With some luck, your LCD1602 should display something like the below. In my case I have two sensors so I’ve fired up the application using the following syntax:

./DS18B20Reader "Sensor1" "Sensor2"

LCD1602OutputCropped

Until the next one!

Using C to monitor temperatures through your DS18B20 thermal sensor – Raspberry Pi Temperature Monitoring Part 3

This is part of a tutorial series. If you feel a bit lost, I suggest following the tutorial in order:

Just interested in the code? – https://github.com/albertherd/DS18B20Reader

Since you made it here, great! Your Raspberry Pi should have one or more DS18B20 thermal sensors connected, like the image below.

sensor2.png

Now that we have our DS18B20 thermal sensor connected to our Raspberry Pi, it’s time to do some programming to read out the temperature! Our application will need to be able to the following tasks:

  1. Discover all the DS18B20 sensors (in my case, I’ve connected 2 but this application should handle an arbitrary number of sensors).
  2. Assign a friendly name so we’ll know which sensor is which and store them in a list.
  3. Retrieve and parse the information from the device.
  4. Do whatever necessary with the gathered information.

1) Discover all the DS18B20 sensors

First and foremost, our code cannot just assume that devices just exist on the system – we’ll need to go and discover these devices. Since the DS18B20 makes use of the 1-Wire protocol, devices will live under the /sys/bus/w1/devices/ directory. Therefore, our code will need to devices living under this directory, whose names start with 28 since all DS18B20 device names will start with 28. Let’s start by knowing how many devices are connected.

typedef struct Sensor
{
    char *SensorName;
    FILE *SensorFile;    
} Sensor;
 
typedef struct SensorList
{
    Sensor **Sensors;
    int SensorCount;
} SensorList;

DIR *dir;
struct dirent *dirEntry;

SensorList *sensorList = malloc(sizeof(SensorList));
sensorList->SensorCount = 0;

if(!(dir = opendir("/sys/bus/w1/devices/")))
    return sensorList;

while((dirEntry = readdir(dir)))
{        
    if(strncmp(dirEntry->d_name, "28", 2) == 0)
    {
        sensorList->SensorCount++;
    }
}

2) Assign a friendly name so we’ll know which sensor is which and store them in a list

Now that we’ve discovered the devices connected to the system, it’s time to save a reference and optionally a friendly name as well. Logic mostly applies from step 1.

sensorList->Sensors = malloc(sizeof(Sensor*) * sensorList->SensorCount);
Sensor **currentSensor = sensorList->Sensors;   

int sensorNamesAllocated = 0;
while((dirEntry = readdir(dir)))
{        
    if(strncmp(dirEntry->d_name, "28", 2) == 0)
    {   
        char *sensorName;
        if(sensorNamesCount > sensorNamesAllocated)
        {
            sensorName = strdup(*sensorNames);
            sensorNames++;
        }
        else
        {
            sensorName = strdup("Sensor");
        }

        char sensorFilePath[64];     
        sprintf(sensorFilePath, "%s%s%s",  "/sys/bus/w1/devices/", dirEntry->d_name, "/w1_slave");
        *currentSensor = GetSensor(sensorFilePath, sensorName);
        currentSensor++;
    }
}

Sensor *GetSensor(char *sensorId, char *sensorName)
{
    Sensor *sensor = malloc(sizeof(Sensor));
    sensor->SensorFile = fopen(sensorId, "r");
    sensor->SensorName = sensorName;
    return sensor;
}    

3) Read the temperature from the device

This is the most exciting part – we actually get to read the temperatures! Using the sensor information we got from steps 1 and 2, we can get the device, open it as a file, extract the readings and parse it accordingly. Using the FILE API makes it very easy to do so – grab all the contents and store it in a buffer. As mentioned in the first tutorial, the content of the file looks as follows –

0b 01 55 05 7f 7e 81 66 bf : crc=bf YES
0b 01 55 05 7f 7e 81 66 bf t=16687

We’re only intereested in the t= component, so some string manipulation and float conversion will take the 16687 and convert it into 16.687C. We’re also doing some range checking since the DS18B20 is rated between -55C and +125C

   
long deviceFileSize;
char *buffer;

FILE *deviceFile = sensor->SensorFile;
fseek(deviceFile, 0, SEEK_END);
deviceFileSize = ftell(deviceFile);
fseek(deviceFile, 0, SEEK_SET);

buffer = calloc(deviceFileSize, sizeof(char));

fread(buffer, sizeof(char), deviceFileSize, deviceFile);
char *temperatureComponent = strstr(buffer, "t=");
if(!temperatureComponent)
{
    free(buffer);
    return -1;
}

temperatureComponent +=2; //move pointer 2 spaces to compensate for t=

float temperatureFloat = atof(temperatureComponent);
temperatureFloat = temperatureFloat / 1000;

if(temperatureFloat  125)
    temperatureFloat = 125;    

free(buffer);
return temperatureFloat;

4) Do whatever necessary with the gathered information

We now have the information at hand, great! We can do many sort of things with it, such as sending an email, activating some other device or whatever is necessary. For demo purposes, we’re simply going to output the contents to the console just to see it working. This is in a loop so we’ll keep reading the temperature until the application exits.

 while(1)
    {
        for(int i = 0; i SensorCount; i++)
        {
            char dateTimeStringBuffer[32];
            strftime(dateTimeStringBuffer, 32, "%Y-%m-%d %H:%M:%S", localtime(¤tTime));

            float temperature = ReadTemperature(sensorList->Sensors[i]);
            printf("%s - %s - %.2fC\n", dateTimeStringBuffer, sensorList->Sensors[i]->SensorName, temperature);
        }       
    }

To try out the code as a whole solution, grab a copy from GitHub – https://github.com/albertherd/DS18B20Reader

In a terminal with git and cmake installed, run the following commands

git clone https://github.com/albertherd/DS18B20Reader
cd ./DS18B20Reader
cmake . && make && ./DS18B20Reader "Sensor"

If the output looks like the below, congratulations!

ds18b20 tutorial sample output.png

In the next tutorial, we’ll pick up from here and we’ll start outputting the content on an LCD1602! Until the next one.

Connecting a LCD1602 with an I2C module to your Raspberry Pi – Raspberry Pi Temperature Monitoring Part 2

The LCD1602 is a very famous LCD that can be connected to various devices such as the Raspberry Pi. The LCD1602 on its own is quite tricky to wire it up since it requires 16 pins to be connected. The LCD1602 can also be purchased with an I2C module, which reduces the amount of pins needed to just 4.

For this tutorial, we’ll be working with a LCD1602 with an I2C module. I got mine from AliExpress for around $2.50. Make sure to grab a set of jumper cables as you’ll need them to connect the LCD to the Raspberry Pi. I got mine from AliExpress as well for around $1.50.

IMG_20190102_110607.jpg

img_20190109_230900

Let’s start by wiring it up. We have 4 pins connect – GND (ground), VCC (power, 5V), SDA (data line) and SCL (clock line). GND and VCC can be connected to any equivalent GND and 5V pin. SDA and SCL should be connected to pins BCM 2 and BCM 3 accordingly.

lcd1602_i2c_raspberrypi

img_20190109_231321

If you’re following the Raspberry Pi Temperature Monitoring Part 1 and connected the DS18B20 temperature sensors, you should now have the following configuration.

lcd1602_i2c_ds18b20_raspberrypi

img_20190109_232649

Great! We’re done from the hardware’s side – let’s start configuring our Raspberry Pi to communicate with our LCD.

Firstly, let’s enable I2C from the Raspberry Pi Config. Fire up the raspi-config to get started: sudo raspi config

Now navigate to Interfacing Options => I2C => Enable I2C

raspi-config-interfacing-options

raspi-config-interfacing-options-i2c

Now that we’ve enabled I2C communication, it’s time to start development! We’ll need to get some tools before we start working though, so fire up a shell and input:

sudo apt-get install i2c-tools.

Once that’s done, the LCD is ready to be programmed! Let’s make sure that the LCD is properly connected and working. In a shell, type:

i2cdetect -y 1.

The output should be something like the below. Note the number outputted by the command; will be needed later on. We’ll need this address when we’re trying our demo code. In this case, the address is “27”.

i2cdetect

Great! Now, it’s time to test out our display and see if it works! We’ll be using a Github library – https://github.com/albertherd/LCD1602. This has been forked from https://github.com/bitbank2/LCD1602. We’ll be using my fork since the original repository has an unresolved issue with clearing the display.

After you’ve cloned the repository in your working directory, it’s time to use the address (27 in my case) obtained earlier. Open the main.c and find the call to lcd1602Init and change second parameter. This is how it looks in my case:

lcd1602Init(1, 0x27);

Now it’s time to compile and run our code. If all goes well, we should be getting some text on the screen. You can change the text to whatever you’d like by changing the following lines in main.c.

lcd1602WriteString("BitBank LCD1602");
lcd1602SetCursor(0,1);
lcd1602WriteString("ENTER to quit");

Build and run using the following commands:

make
make -f make_demo
./demo

The screen should look like the below:

IMG_20190110_231937.jpg

Great! Now we’ve successfully connected our LCD1602 to our Raspberry Pi and we’re able to output content on it!

In the next part of this tutorial series, we’ll start by capturing the temperature using the sensor in our first part of the tutorial and outputting it! Stay tuned.

Connecting a DS18B20 thermal sensor to your Raspberry Pi – Raspberry Pi Temperature Monitoring Part 1

A project that I’ve been working on during the Christmas holidays was to hook up some thermal probes to my Raspberry Pi, just to play around. This tutorial simply follows the steps that I’ve taken to achieve so.

You’ll need:

  • Raspberry Pi, any flavor as long as it has GPIO headers available. I had a Raspberry Pi 2, so I used that.
  • You’ll also need the usual suspects – USB to MicroUSB to hook it up to power, HDMI to connect it to a display for initial configuration and an ethernet port to manage it through SSH. I highly recommend configuring SSH rather than using the device itself. This tutorial assumes you’re using SSH.
  • A DS18B20 sensor – I’d suggest getting one which includes a Plugable Terminal to avoid soldering – just wire it up and you’re good to go. I got mine from AliExpress
  • Also make sure your kit has 3 jumper cables. They are typically included. Just to be sure, I also got a set of female to female jumper cables from AliExpress though I did not use them for the DS18B20 sensor.

All right, let’s wire it up! The DS18B20 sensor requires three pins – data, VCC (3.3V), and ground. Connect the wires as below. Data is yellow, VCC is red and ground is black.

IMG_20190102_105252

Connect the 3 pins using the jumper cables as shown below.
sensor1.png

IMG_20190102_105726

We’ll also need to instruct the Raspberry Pi that we’re going to connect the DS18B20 sensor. This sensor makes use of the 1-Wire protocol, so let’s activate it:

  • Connect to the Raspberry Pi using SSH
  • Let’s start by editing the config file that the Raspberry PI parses every time it boots up: sudo nano /boot/config.txt
  • Go to the end of the document and input the following. Specifying gpiopin=4 is actually optional since by convention, 1-wire devices are expected on gpiopin 4 on the Raspberry Pi.
    # Enable OneWire Protocol
    dtoverlay=w1-gpio;gpiopin=4
  • Time to reboot the Raspberry Pi sudo reboot
  • Once the Raspberry PI reboots and you re-connect using SSH, it’s time to get data from the sensor! Let’s find the 1-wire devices connected to the system. Let’s start by browsing to the appropriate directory. cd /sys/bus/w1/devices
  • Great! Let’s now see the devices attached to the Raspberry Pi. ls
  • This will get the devices attached using the 1-Wire protocol. You should have a device called 28-xxxxxxxxxxxx (where x stands for your unique 12 digit serial number). Let’s now browse the device. Mine is 28-02199245e07b, so let’s use it an example. cd 28-02199245e07b
  • Once you access the device, there should be a file called w1_slave. Let’s see the contents of the file. cat w1_slave
  • The file should look like this:
    0b 01 55 05 7f 7e 81 66 bf : crc=bf YES
    0b 01 55 05 7f 7e 81 66 bf t=16687
  • If the file looks like the above, great! The temperature component is t=16687. The temperature in this case is 16.687 °C

We also can take this to the next level and add another thermal probe! Attach it as shown below.
sensor2IMG_20190102_110134

This will require re-editing the /boot/config.txt. Let’s do it!

  • Re-open /boot/config.txt – sudo nano /boot/config.txt
  • Go to the end and add the following. I chose pin 24 because it’s easy to wire since it’s close to a 3.3v and ground. dtoverlay=w1-gpio;gpiopin=24
  • Close and save, then cd /sys/bus/w1/devices
  • You should now see two devices as 28-xxxxxxxxxxxx

Of course, at this stage we did get the temperature, but it’s not really usable. We can get access to this information programmatically – this is what we’ll be doing in the next part of this tutorial. We’ll also be eventually showing the information on a separate LCD screen! Stay tuned!

Group files into folder structure by date using PowerShell

Lately I was sorting out around 5000 files worth of pictures and videos taken out from a mobile phone, to upload them on a cloud drive. To make this task more manageable, I wanted to separate these files out, firstly by file type and then by date. This will make the folders far more smaller and therefore easier to upload these files to the cloud folder by folder.

Grouping such several files manually will be very tedious and error-prone. This task is a perfect candidate to be scripted out; this is exactly what I did. Since this may be helpful to other people going through such task, I decided to share this script.

All the script requires is a source path and a target path – anything else is handled by the script. It has several assumptions, such as it always moves, never deletes a file. Customising the script is easy and only requires basic programming knowledge.

Check it out here or below: https://github.com/albertherd/GroupFilesPowershell/blob/master/GroupFiles.ps1

Param(
    [Parameter(Mandatory=$true)]
    [string]$sourcePath,
    [Parameter(Mandatory=$true)]
    [string]$targetPath
)

$global:fileTypeLookup = @{};
$folderDateTimeFormat = "MM-yyyy"

function Copy-FilesIntoFoldersByMonthAndType{
    param()

    $files = Get-ChildItem -Recurse -File -Path $sourcePath
    $filesProcessed = 0

    foreach($file in $files){
        $folder = Get-DirectoryForFile $file
        Copy-Item $file.FullName -Destination $folder

        $filesProcessed++
        Write-Progress -Activity "Grouping files" -Status "$($filesProcessed) out of $($files.Count) grouped" -PercentComplete (($filesProcessed / $files.Count) * 100)
    }
}
function Get-DirectoryForFile{
    param($file)

    $monthYearDirLookup = Get-FilePathDictionary $file
    $modifiedTimeMonthYearInternal = $file.LastWriteTime.ToString("MMyyyy")

    if($monthYearDirLookup.ContainsKey($modifiedTimeMonthYearInternal)){
        return $monthYearDirLookup[$modifiedTimeMonthYearInternal]
    }

    $extensionWithoutDot = $file.Extension.Substring(1, $file.Extension.Length - 1)
    $dateFolderFileName = $file.LastWriteTime.ToString($folderDateTimeFormat)
    $newPath = $targetPath + "\" + $extensionWithoutDot + "\" + $dateFolderFileName
    $path = New-Item -ItemType Directory $newPath -Force

    $monthYearDirLookup[$modifiedTimeMonthYearInternal] = $path.FullName
    return $path.FullName
}

function Get-FilePathDictionary{
    param($file)

    if($global:fileTypeLookup.ContainsKey($file.Extension)){
        return $global:fileTypeLookup[$file.Extension]
    }

    $global:fileTypeLookup.Add($file.Extension, @{})
    return $global:fileTypeLookup[$file.Extension]
}

Copy-FilesIntoFoldersByMonthAndType

In my case, it generated the below folder structure:

FolderStucture

Far more manageable than one flat folder of 38.8 GB folder, for sure!

Until the next one.

GDPR and the Maltese Political Propaganda

Unless you’ve been living under a rock, GDPR is now in effect as of 25th May 2018. I want to discuss the impact of GDPR, with regards to the Maltese Political Propaganda. This new law brings up a lot of changes and rights to the end user – the below is a summary which should be easily understandable without any jargon.

Politicians CAN send you personalized letters through postal services

Politicians do have the right to obtain your home address through three main channels: Electoral Register, Online Directory and Printed Directory. Though, these letters must have clear instructions on how one can easily opt out of these personalized letters and they must provide a method to do so, such as a mailing address, email or contact number.

Politicians CAN call you on your telephone / mobiles

Politicians do have the right to obtain your telephone / mobile again through three main channels: Electoral Register, Online Directory and Printed Directory. Though, the caller must respect the fact that you can ask them to hang up and remove your personal details so that they do not contact you again. One can also opt out of the directories in order to hide their telephone / mobile number, through their phone operator.

Politicians CAN contact you on election day to encourage you to go out and vote

Politicians can contact you using different mediums, which may be obtained from the following: Online Directory, Printed Directory public Social Media information or any publicly made information that the end user has made public himself. Of course, you can ask whoever is contact you to stop any communications and immediately delete your personal information.

Politicians CANNOT send you e-mails UNLESS you have given them an explicit consent and your email directly

In order for a politician to obtain your e-mail address, you must give it to them. This is typically done through direct political channels, such as propaganda websites. Also, these channels must provide an easy way to opt out of these emails, such as a unsubscribe link, which is clearly labelled and visible appopriatelty. The unsubscribe link must be send with each and every email.

Politicians CANNOT send you SMSs UNLESS you have given them an explicit consent and your phone number directly

In order for a politician to obtain your phone number, you must give it to them, exactly like the previous case with e-mails. With each SMS that the subscribed user receives, the sender must provide a contact number or email with each and every SMS sent, which the receiver can use in order to opt out of such SMSs.

3rd Party contractors which may send Propaganda SMSs (such as Go / Vodafone) MUST ensure that the politican has obtained the proper consent of the end user

If the politician does not provide clear and legitamate proof of consent to the 3rd Party contractor, the contractor must refuse to send propaganda SMSs to end users.

Liking / Following a politicians’ account on Social Media means that you’re giving him consent in order to obtain your PUBLIC Social Media information, such as e-mail or phone number.

Any information that you publicly post on Social Media can be accessed and used for political propaganda, by simply liking / following a page / account.

Some additional information:

  1. You ALWAYS have the right to opt-out of any propaganda messages, whether’s it’s postal, email, SMS or any other means, even if it’s something that has not been invented yet.
  2. Consent is ALWAYS needed for electronical communications, whether it’s email, or some futuristic electronic holographic message, consent is ALWAYS needed.
  3. Personalized letters can be sent through any means, be it postal services, or some fancy pigeon delivery. Keep in mind that you can always opt out of these letters, but by default, you’re in.
  4. If you’re tired of receving door-to-door political propaganda, you can stick a “no junk mail” message with your letterbox, and they must respect this fact.
  5. Politicians CANNOT charge you money to opt you out of any communication

Note that I’ve omitted some less important information in this article, such as dealing with Automated Calling Machines. All this information has been obtained from the Office of the Information and Data Protection Commissioner. If you feel that your rights have been violated, you can submit a report here.
Note by all means this article is not a legal advice, merely a guideline on your rights.

Source of article is here