Padding is done to keep members of the struct aligned, so that access is fast.(*) Consider what happens in an array of structs:
struct s {
int i;
char c;
};
struct s a[0];
If this struct were not padded, a[1] would have the address (char *)a + 5. That's unaligned, assuming a[0] is aligned. Therefore, the struct will typically be padded to a multiple of sizeof(int). (Were i a long long, then the whole struct would be padded to a multiple of sizeof(long long); try and see.)
With a single char, there's no need to pad because the char will be aligned anyway, while there are no other members that become unaligned. With a single short, an alignment of 2 is sufficient to keep the single member aligned, always.
(*) On some CPUs, but not Intel x86-compatible ones, unaligned access can actually crash your program.