Pointers: The why

I recently had a small discussion with a friend of mine who is pursuing a Computer Science education. It was mostly about pointers and void pointers, what they are and how they work along with some examples. However, when discussing this, I noticed that even throughout my personal journey of self-learning, it wasn't that easy to find resources explaining the WHY of pointers, instead, most of them were focused on explaining the WHAT and HOW of this seemingly complex topic. Thus, what I will be trying to do in this article is to explain WHY the pointer type exists, what advantages does it bring and what would happen if we didn't have pointers. Hope this will be of any help to anyone reading. Let's go...

Prerequisits: This article expects a basic familiarity and understanding of C and pointers.

Are pointers really needed?

This question is not as silly as it looks like. While we are all taught that pointers store the address of “something”, the question of why we need a pointer to do that in the first place is totally legit. For instance what is wrong with this code: 

int main() {
    int num = 5;
    int numPtr = #
}

Some people will say, this doesn't work because pointers can be 8 bytes (on modern systems) not 4 bytes, and they are positive numbers always. Ok fine, let's adjust our code then: 

      int main() {
    int num = 5;
    unsigned long numPtr = #
}

Now we have 8 bytes reserved to store the address of num, does this work? In order to test it and compare it to pointers, let me adjust the code one last time so we can do a proper comparison: 

#include <stdio.h>

int main() {
    int              num = 5;
    unsigned long    fakePtr = &num;
    int             *realPtr = &num;

    printf("The address of num is: %lx\n", fakePtr);
    printf("The address of num is: %p\n", realPtr);
}

Now when we compile and run our program, we get the following output:

$ ./main
The address of num is: 7ffccb97c884
The address of num is: 0x7ffccb97c884

It works! using a regular long variable to store the address of num works fine without needing any pointer type. So the question is still valid; What is the point of pointers? Let's go for a different example and test our pointer-less pointer invention and see if it works. 

#include <stdio.h>

int main() {
    int              nums[] = {8, 10, 15};
    unsigned long    fakePtr = nums;
    int              *realPtr = nums;

    printf("The address of num is: %lx\n", fakePtr);
    printf("The address of num is: %p\n", realPtr);
}

Note:

nums is in itself the address of where the array starts in the memory, but for the sake of keeping things simple, let’s just store this address in a pointer variable for demonstration purposes.


And the output is:

$ ./main
The address of nums is: 7ffc3ea5eae4
The address of nums is: 0x7ffc3ea5eae4

No surprises so far and our fake pointer still works fine for arrays too!


Knowledge check:

As you might already know, when you access an array element with the syntax nums[1], this index is actually the offset and is a syntactic sugar on top of the following syntax: *(nums + 1) which in simpler words means:


Now, based on the small pause above, we need to double check that both of our pointers still behave identically, so let's check the address of the second element in the array: 

#include <stdio.h>

int main() {
    int              nums[] = {8, 10, 15};
    unsigned long    fakePtr = nums;
    int              *realPtr = nums;

    printf("1st num is at: %lx -- 2nd num is at: %lx\n", fakePtr, fakePtr + 1);
    printf("1st num is at: %p -- 2nd num is at: %p\n", realPtr, realPtr + 1);
}

And the output is:

$ ./main
1st num is at: 7fffef6da2bc -- 2nd num is at: 7fffef6da2bd
1st num is at: 0x7fffef6da2bc -- 2nd num is at: 0x7fffef6da2c0

Hmm … Things are differenet now! The address of the first element is the same for both, but the address of the second one is different. What's happening?

Moment of truth: Pointers don't just 'store' addresses

Even though pointers are generally defined as variables that contain the address of another variable, the story doesn't end here, and the benefits of pointers are way beyond just storing addresses, since as we have seen so far, the act of storing can be easily done with a regular unsigned long variable.

Let's rewind to where we left off and try to understand what happened. When we added 1 to fakePtr, the compiler literally did a simple arithmetic operation on the hex value and added 1 to 7fffef6da2bc which led to: 7fffef6da2bd. However when we added 1 to a pointer variable, something else happened! It actually added 4 bytes! Why 4 bytes? because of how the pointer was initially declared: 

int   *realPtr = nums;

On this line, we specified that realPtr points to values of type int, so the compiler automatically knows that when we do an arithmetic operation on a pointer like realPtr + 1, we don't only add 1, but we also take into account the size of the type this pointer is referring to, which leads back to the knowledge check discussed above:

realPTr + 1 is in fact: realPtr + 1 * sizeof(int).

We can verify this by taking a look at the assembly code for both printf statements: 

16. mov rax, QWORD PTR [rbp-8]
17. add rax, 1                  <-- We add 1 
18. mov rsi, rax
19. mov edi, OFFSET FLAT:.LC0
20. mov eax, 0
21. call printf

22. mov rax, QWORD PTR [rbp-16]
23. add rax, 4                  <-- We add 4
24. mov rsi, rax
25. mov edi, OFFSET FLAT:.LC1
26. mov eax, 0
27. call printf

In the first call to printf, we store the hex value at [rbp-8] in rax and then add 1 to it on line 17. However in the second call, we actually add 4 as you can see on line 23. So it's the pointer type that gives us this implicit mechanism for free, which allows us to use arrays and traverse the elements using offsets/indexes.

Casting void pointers: Why?

All this long discussion and we haven't mentioned void pointers yet! How does all this relate to void pointers and casting them? Well, now that we have introduced the building block of why pointers are important and needed, we can take a look at the following code snippet: 

#include <stdio.h>

int main() {
    unsigned int    num = 511;
    void            *ptr = &num;
}

In this code, we are storing the int 511 in num, and then storing the address of num in ptr which is declared as a void pointer. The idea of void pointers is simpler than it looks like most of the times: A void pointer is a pointer that is not associated to any type yet. Why? well, to simulate the behavior of generic functions for example. Sometimes you want to define a function that can work for different types and not just only one. The most obvious example is malloc which has the following prototype: 

void *malloc(size_t size);

malloc returns a void pointer, and thanks to this, you can allocate memory to any type you want. Now the question is, why do we need to cast this void pointer? If all pointers have the same size of 8 bytes, and we are not doing any arithmetic operations on the pointer, would our good old fake pointer be enough? …well…. let's revise the code snippet above and see how the number is represented in memory.

And this will be stored as follows:

--------------
  0000 0000            ---> Fourth byte in memory   
--------------
  0000 0000            ---> Third byte in memory
--------------
  0000 0001            ---> Second byte in memory   
--------------
  1111 1111            ---> First byte in memory
--------------

Let's try to go and read the integer via the void ptr:

#include <stdio.h>

int main() {
    unsigned int    num = 511;
    void            *ptr = &num;

    printf("Num is %d\n", *ptr);
}

When running this code we get the following error:

main.c:8:27: warning: dereferencing 'void *' pointer
    8 |     printf("Num is %d\n", *ptr);
      |                           ^~~~
main.c:8:27: error: invalid use of void expression

So what is the problem exactly? As we have stated in the previous section, a pointer is tightly associated with the type it points to. The relation between them is so important that the compiler cannot compile your code if you don't specify what this pointer points to. But why? Simply put: The compiler needs to know how many bytes to read!

Not specifying the number of bytes the computer should read can lead to incorrect results. Let's try the following code: 

int main() {
    unsigned int    num = 511;
    void            *ptr = &num;

    printf("Num is %d\n", *(char *)ptr);
}

We casted the void pointer to a char pointer. What this basically does is transform the initial declaration of the void pointer from: void *ptr to char *ptr, and then we dereference it which gives us the following output: 

$ ./main
Num is -1

Hmm…what is happening here? Some of you might have expected 511, and others might have expected 255, and I wonder how many anticipated -1. Leave a comment if you saw -1 coming!! (No I'm kidding there is no comment section here.)

The topic of why -1 is the result is beyond the scope of this article, but basically it is related to computers interpreting numbers using two's complement unless specified otherwise. So let's simplify things and specify that we are dealing with unsigned numbers for now: 

int main() {
    unsigned int    num = 511;
    void            *ptr = &num;

    printf("Num is %d\n", *(unsigned char *)ptr);
}

and now the output is:

$ ./main
Num is 255

So why 255? what happened? Well, if we take a look again at the memory layout we have above, we see that the first byte is 1111 1111 which is the equivalent of 255 in base 10. This happened because of our casting! We told the compiler to look at the addresses pointed out by ptr and read only one byte! How? By casting it into a char pointer, and since char type is only one byte in C, it will only read one byte!

So now we know that pointers help with:

- Storing addresses

- Reading

- Doing some magical arithmetics

And how do we fix the problem we have? We cast it to an int pointer:

int main() {
    unsigned int    num = 511;
    void            *ptr = &num;

    printf("Num is %d\n", *(unsigned int *)ptr);
}

Which will transform the void pointer into an int pointer that reads 4 bytes at a time and gives the following output: 

$ ./main
Num is 511

If you want to play around a bit, you can even try to cast it into a short pointer, which will be fine for any number that fits into two bytes, like our 511 example.


The above explains that pointers not only help with storing addresses, but also they indicate to the compiler when dereferencing them, how many bytes are supposed to be read in order to get the value we want. Now we also understand why we need to cast the pointer returned by malloc, and hopefully have a better understanding of what pointers help with. The relation between the pointer and the type it points to is what allows many of the things we take for granted to happen correctly.

Summary

There is a lot more to say about pointers and how they are used, but everything boils down to the original question of this article. Can storing the address in a regular variable be enough? The simple answer is no for a combination of different reasons:

For all these reasons and probably more, we can easily conclude that having a separate type that handles memory addresess exclusively is a better solution to all our problems.

Hope you learned something from this article, until next time...Keep coding! :)