What's the unit of partitions of hard disks?

As you can see e.g. on Wikipedia, the entries in the GPT are LBA's (Logical Block Addresses). These are addresses based on the logical sector size.

When harddisks started to have a different (larger) physical sector size, the vendors kept the old physical sector size as logical sector size, to help old systems use the new harddisks. This means the harddisk firmware contains code that can deal with logical sector reads and writes: For a read, it will read the correct (larger) physical sector and return only part of it. For a write, it will first read the (larger) physical sector, then modify part of it, and the move it back.

This is inefficient unless reads and writes happen at LBAs that correspond to the start of a physical sector, and with lengths that are multiple of physical sectors.

So the way the kernel deals with large physical sector sizes is that it tries only to use these kind of reads and writes. For this to work, the partitions must be aligned correctly (on boundaries of physical sectors). Usually, partitioning programs will ensure that this is the case, and print a warning if you try to use them in a way where it isn't.

The kernel doesn't have to do that; it would work even without doing it, but it would be less efficient.

Edit

Can userland apps just read the logical sector size through /sys and read the partition table then multiply them together to tell the user how big a partition is?

Yes, userland apps like fdisk do exactly that (though I think the physical and logical sector size is in /proc, not in /sys, but I'd have to look this up).

When the kernel operates filesystem, it doesn't need to know the size of underlying storage, just the size of filesystem structure by reading the metadata of the filesystem, is that right?

It's a bit more difficult: The basic kernel unit is a "page" (the MMU unit), and the basic unit of a particular filesystem is a "block" (sometimes multiple pages). The kernel just needs to know how this relates to the logical block size of the underlying storage, because that's needed to calculate the LBAs.

But kernel needs to know the start and size of a partition when userland access raw partition device like /dev/sda2, isn't that true?

Yes. That's why it reads the partition table before it creates entries like /dev/sda2.

When kernel do this, it needs to know the unit of numbers recorded in partition table is logical sector size, not physical sector size, right?

Yes.