I was recently building a NiFi Flow for CloudTrail events that enriched the events with IP geolocation data, then wrote them to an S3 bucket to query with Athena. But I wondered, is it possible to use Athena to query CloudTrail records directly from S3 without reprocessing them?

The answer is yes, as long as some tortured SQL syntax doesn't bother you.

I have git repos on both GitHub and AWS CodeCommit, but I found CodeCommit's HTTPS credential management to be a bit problematic. CodeCommit's credential helper does not follow the typical name/password pattern, and the default git credential helpers installed for both Windows and OSX do not naturally play nice side-by-side with CodeCommit.

1. I followed Amazon's documentation for setting up CodeCommit's credential helper and overwrote my GitHub-compliant credential helper configuration.
2. I tried to configure them side-by-side with CodeCommit's helper namespaced to the CodeCommit HTTPS domain. I learned about git credential helper configuration settings figuring out the earlier problem of getting CodeCommit to work with EC2 Role Credentials. So I felt really smart, and I was proud of myself for figuring out domain scoping of credential helpers for a few minutes -- until this stopped working because credential helpers are executed in a cascading chain.
3. Finally, I configured both helpers by HTTPS URL scope, so they play nicely side-by-side.

Yay.

Amazon's new API Gateway service has great potential, probably an avenue for future expansion of AWS, and certainly something I'm trying to get up to speed on. But API Gateway definitely has some quirks.

Today, I ran into a strange aspect of the permissions model for API Gateway while answering a StackOverflow question about why the documented permissions didn't work in the IAM Policy Simulator , and it's still bothering me. Amazon made an intersting choice with the permissions model that seems consfusing to developers and fateful with respect to future services. In short, I would call it "special", for not being with the same program every other AWS service used to define their permissions.

The bottom line is that API Gateway has its client and admin/management permissions broken out under different service names. When you look at the list of services for permissions, you see:

• API Gateway - Permissions for clients, currently the only action is `execute-api:invoke`.
• Manage - API Gateway - Admin permissions for configuring the API Gateway, which has CRUD actions fitting the `apigateway:*` spec.

I had trouble getting AWS CodeCommit to work using an EC2 instance's role/instance profile credentials. My goal was to use a cloud-init script to pull down source from CodeCommit at instance launch time, then kick off some processing. I spent more time that I wanted to struggling with CodeCommit credentials, and I'd like to share the results in the hope of saving someone else the anguish.

My initial desire to use CodeCommit rather than github was that it used IAM credentials. An EC2 instance with credentials from an IAM Role seems like a pretty vanilla use case to me, but is not covered in the CodeCommit docs for setting up a git client. The documentation focuses on interactive or desktop use rather than automated agent access.

TL;DR

There were three configuration challenges and one obvious task:

1. I ran into errors running git as root from the cloud-init script. I used git system-wide seetings as a workaround.
2. The AWS CodeCommit credential helper for git must be configured with the "default" named profile to use the instance's role/instance profile credentials.
3. The "default" named profile must be set to use the us-east-1 region by default, to match CodeCommit.
4. The EC2 role needs to have permissions to the CodeCommit repo.

Below, I'll explain the setup in more detail and show a sample cloud-init script that puts it in action.

Recently, AWS announced support for "periodic worker tasks”, a.k.a. cron jobs, a.k.a. scheduled tasks in ElasticBeanstalk worker tiers. This is a feature I’ve been missing in Elastic Beanstalk, and I was unjustifiably excited when I read the announcement. This post is a record of my thrilling journey to setting up and using a periodic worker task. I had three problems:

1. Periodic Worker Tasks are only supported on the Worker tier of Elastic Beanstalk
2. cron.yaml is not documented by AWS (yet)
3. Exactly how my application code would be invoked by the task was not clear