a is an int*, not an int. What you are printing is actually the pointer, i.e. the memory address of the pointed object. Use the dereference operator * whenever you want to modify the pointed value, i.e.
cout << (*a)++ << endl;
NB: Likewise, you can get a pointer to an int using the reference operator, &, not to be mixed up with a reference (e.g. a int& type).
This may print 0 1 2 3 4. may because you are not initializing the new int created in dynamic memory. This means reading from *a (dereferenced a) is undefined behavior, which means your program may misbehave. You have to change your line using new:
int *a = new int();
This will initialize *a to 0 and now 0 1 2 3 4 will be printed correctly.
Note that int *a = new int[2]; does create a dynamic array of 2 entries in dynamic memory, which means *(a + 1) can be used as well (as if it was a regular array). It does not initialize *a to 2.
Do remember to delete a; when you've done using it. In a real application, you could get a memory leak if you don't - i.e. your program would still use memory it doesn't need anymore. Caution, when you have to delete a dynamically-allocated array (i.e. new int[2]), you need to use delete[] a; instead, or you will trigger undefined behavior.
You may also use a unique_ptr (or a shared_ptr) in C++11 as an alternative to this kind of memory allocation, i.e. :
#include <memory>
// ...
std::unique_ptr<int> a = std::make_unique<int>(0);
Thanks to this solution, you do not need to delete a because the unique_ptr will do this for you, when itself dies (i.e. out of the scope, here).
Edit: Bonus:
0x2518c20
0x2518c24
0x2518c28
Why is the number incremented by 4 if you just used ++?
Using ++ on an address will actually increment it by sizeof(T), which here is sizeof(int) and not 1. This explains why you can, as previously stated, use *(a + 1) if you used new int[2].