Listing Folders on S3
Just Trying to Parse and Display a Whole Bucket, I Can't Get Rid off the Message No suchKeyException Which Is Odd. . the Code Below Is Connecting and...
just trying to parse and display a whole bucket, i can't get rid off the message no suchKeyException which is odd ..
The code below is connecting and displaying to standard out:
import software.amazon.awssdk.auth.credentials.ProfileCredentialsProvider
import software.amazon.awssdk.regions.Region
import software.amazon.awssdk.services.s3.S3Client
import software.amazon.awssdk.services.s3.model.EncodingType
import software.amazon.awssdk.services.s3.model.ListObjectsV2Request
import software.amazon.awssdk.services.s3.model.ListObjectsV2Response
import software.amazon.awssdk.services.s3.paginators.ListObjectsV2Iterable
import java.net.URI
internal class S3ObjectsOps {
companion object {
private val BUCKET: String = "mybucket"
private val REGION_IRE = software.amazon.awssdk.regions.Region.EU_WEST_1
val s3c = S3Client.builder()
.region(REGION_IRE)
.endpointOverride(URI(""))
.credentialsProvider(ProfileCredentialsProvider.builder()
.profileName("default")
.build())
.build()
fun lsfolders(bucket: String) {
val listReq = ListObjectsV2Request.builder()
.bucket(bucket)
.delimiter("/")
.prefix("")
.maxKeys(10_000)
.build()
val listRes = s3c.listObjectsV2Paginator(listReq)
listRes.contents().onEach { f ->
if (!f.key().isNullOrBlank()) {
println(">> ${f.key()}")
} else {
println(" = ")
}
}
}
@JvmStatic
fun main(args: Array<String>) {
val bucket: String = ((args.size > 0 && !args[0].isNullOrEmpty()).toString()) ?: BUCKET
println("s3 ls test for bucket ${bucket}")
lsfolders(bucket)
}
}
}
Which raises:
software.amazon.awssdk.services.s3.model.NoSuchKeyException: The specified key does not exist. (Service: S3Client; Status Code: 404; Request ID: C8AF9CB788D77F74) Exception in thread "main" software.amazon.awssdk.services.s3.model.NoSuchKeyException: The specified key does not exist. (Service: S3Client; Status Code: 404; Request ID: C8AF9CB788D77F74) at software.amazon.awssdk.core.http.pipeline.stages.HandleResponseStage.handleErrorResponse(HandleResponseStage.java:114)
Same Exception using a stream
listRes.contents().stream().forEach { content -> println(" Key: " + content.key() + " | ") }
Thanks folks !
1 Answer
The right calls of ListObjectsV2Request is:
val listReq = ListObjectsV2Request.builder()
.bucket(bucket)
.delimiter("/")
.build()
val listRes = s3c.listObjectsV2Paginator(listReq)
listRes.contents().stream().forEach { content -> println(" Key: " + content.key() + " | ") }
Then, s3client must init with mimimum params:
val s3c = S3Client.builder()
.credentialsProvider(ProfileCredentialsProvider.builder().profileName("default").build())
.region(REGION_IRE)
.build()