Linux deletes a file completely differently than it is doing Windows. First a brief explanation on how files are managed in the *unix native file systems.
The file is kept on the disk in the multilevel structure called `i-node`. Each i-node has uniq number on the single disk.The i-node structure keeps different information about file, like its size, data blocks allocated for the file e.t.c, but for the sake of this answer the most important data element is a `link counter`. The `directories` are the files that are keeping records about the files. Each record has the i-node number it refers to, the file name length and the file name itself. This scheme allows one to have a 'pointers', i.e. 'links' to the same file in different places with different names. The link counter of the i-node actually keeps the number of links that refers to this i-node.

What happens when some process opens the file? First the `open()` function searches for the file record. Then it checks if the in-memory i-node structure for this i-node already exist. This may have place when some application having this file opened. Otherwise, the system initializes a new in-memory i-node structure. Then system increases the in-memory i-node structure open counter and returns to the application it's file descriptor.

 The Linux library call to delete a file is called `unlink`. This function removes the file record from a directory and decrements the i-node's link counter. If system found that in-memory i-node structure exist and it's open counter is not zero then this call returns the control to the application. Otherwise it checks if the link-counter became zero and if it does then the system frees all blocks allocated for the i-node and the i-node itself and returns to the application.

What happens that an application closes a file? The function `close()` decrements open counter and checks its value. If the value is non-zero the function returns to the application. Otherwise it checks if the i-node link counter is zero. if it is not zero - it returns to the application or frees all blocks of the file and the i-node.

This mechanism allows to "delete" a file while it is opened. At the same time the application that opened a file still has an access to the data in the file. So, JRE, in your example, still keeps its version of file opened while there is another, updated version exist on the disk.

More over, this feature allows to update the glibc(libc) - the core library of all applications - in your system not interrupting it's normal operation.


Windows. 
The 20 years ago we did not know any other file system than FAT under DOS. This file system has a different structure and management principles. These principles does not allow to delete file when it is opened, so the DOS and lately Windows has to deny any delete requests on a file that is open. Probably, NTFS would allow the same behavior as *nix file systems but Microsoft decided to maintain the habitual behavior of the file deletion. 

This is the answer. Not short, but now you have the idea.