構成
Glue Crawlerを用いてS3バケット上のファイルからData Catalog (Database)を作成する構成です。
IAMポリシー
AWS Glue Crawlerに必要なIAMポリシー
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:ListBucket"
],
"Resource": [
"arn:aws:s3:::bucket_name"
],
"Condition": {
"StringLike": {
"s3:prefix": "prefix/*"
}
}
},
{
"Effect": "Allow",
"Action": [
"s3:GetObject",
"s3:PutObject"
],
"Resource": [
"arn:aws:s3:::bucket_name/prefix/*"
]
},
{
"Effect": "Allow",
"Action": [
"glue:GetDatabase",
"glue:GetTable",
"glue:CreateDatabase",
"glue:CreateTable"
],
"Resource": [
"arn:aws:glue:region:account-id:catalog",
"arn:aws:glue:region:account-id:database/database_name",
"arn:aws:glue:region:account-id:table/database_name/*"
]
},
{
"Effect": "Allow",
"Action": [
"logs:CreateLogGroup"
],
"Resource": [
"arn:aws:logs:region:account-id:log-group:/aws-glue/crawlers:*"
]
},
{
"Effect": "Allow",
"Action": [
"logs:CreateLogStream",
"logs:PutLogEvents"
],
"Resource": [
"arn:aws:logs:region:account-id:log-group:/aws-glue/crawlers:log-stream:*"
]
}
]
}
参考
AWS Glue のアイデンティティとアクセスの管理 - AWS Glue
AWS Glue リソースへのリクエストを認証し、アクセスを管理する方法について説明します。