5

I have a S3 bucket with following hierarchy:

bucketName
    folder1
       file1

I wanted to get all the files from folder1. I tried to do following:

ObjectListing ol = s3Client.listObjects("bucketName", "folder1"); 
List<S3ObjectSummary> summaries = ol.getObjectSummaries();

The problem is that summaries contains folder1/ and folder1/file1. Where as I was hoping to get just folder1/file1.

Looking around at the internet, I also tried following:

ListObjectsRequest req = new ListObjectsRequest().withBucketBucketName("bucketName").withPrefix("folder1/").withDelimiter("/");

But this time I got no results back for getObjectSummaries call. When I remove withDelimiter from above I get both folder1\ and folder1\file1 back.

Is there any way to just get folder1\file1 back?

starball
  • 20,030
  • 7
  • 43
  • 238
test123
  • 13,865
  • 9
  • 28
  • 33

1 Answers1

7

Use withPrefix and withMarker together:

ListObjectsRequest req = new ListObjectsRequest().withBucketName("bucketName").withPrefix("folder1/").withMarker("folder1/");

This works because first you filter withPrefix and obtain all folder1/* keys, including folder1/, and then with withMarker("folder1/") you specify to get the keys that are lexicographically after "folder1/" as documented in the javadoc:

The list will only include keys that occur lexicographically after the marker.

Furthermore, if folder1 contains other subfolders, you can get the direct children only, using withDelimiter:

ListObjectsRequest req = new ListObjectsRequest().withBucketName("bucketName").withPrefix("folder1/").withMarker("folder1/").withDelimiter("/");

This works because the delimiter "/" makes all subfolders to be rolled up to "folder1/" but you ignore this result with the marker. The javadoc says for withDelimiter:

Gets the optional delimiter parameter that causes keys that contain the same string between the prefix and the first occurrence of the delimiter to be combined into a single result element (...). The most commonly used delimiter is "/", which simulates a hierarchical organization similar to a file system directory structure.

--

In any case, folder1/ is only listed because you surely created it through the web console. If you don't create folders directly but put objects programmatically, like put folder2/file2 the folder won't be actually created as an independent object and so it cannot be listed.

juanmirocks
  • 4,786
  • 5
  • 46
  • 46
  • It ended up returning all the folders with their contents that were at the same level. So for ex., if there was `folder2` at the same level as `folder1` then it returned `folder2` as well. Anyways, I ended up using, `FileNameUtils.getName()` to filter out `folder1` from the results set. – test123 Apr 13 '13 at 22:09
  • @test123, doing this you will get all the folders from the s3 location. And you are then filtering it. Which i think is not a feasible solution. In my case I would want to get all files in folder1 but not from folder1/subfolder1. Is there any way to do it directly while fetching the s3objects – mwKART Dec 08 '20 at 10:24