You need to set at least the ETag, Last-Modified and Expires headers on the response. The ETag header should represent the unique identifier of the file based on a combination of filename, filesize and lastmodified timestamp). The Last-Modified header should represent the last modified timestamp of the file. The Expires header should inform the client how long it is allowed to keep the file in cache.
If the cache has been expired in the client side and the ETag or Last-Modified are available, then the client will send a HEAD request to check if the file needs to be renewed. If not, then the Expires will just be postponed again accordingly.
You can find here a servlet example which handles this all (and download resumes and automatic GZIP): FileServlet supporting resume and GZIP.
However, if your files are available on the disk file system already (and not in a database), then you should consider just delegating the job to the servletcontainer's builtin default servlet. See also Reliable data serving.