|
Size: 178
Comment:
|
Size: 5800
Comment:
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 1: | Line 1: |
| <<TableOfContents(2)>> |
|
| Line 4: | Line 6: |
| Lambda Java samples: * [[ https://docs.aws.amazon.com/lambda/latest/dg/lambda-samples.html | lambda samles ]] * [[ https://github.com/awsdocs/aws-lambda-developer-guide/tree/main/sample-apps/java-basic/ |sample apps java basic ]] == User credentials == * [[ https://docs.aws.amazon.com/general/latest/gr/root-vs-iam.html | root vs iam ]] * [[ https://docs.aws.amazon.com/general/latest/gr/aws-access-keys-best-practices.html | aws access keys best practices ]] * [[ https://docs.aws.amazon.com/IAM/latest/UserGuide/best-practices.html | IAM best practices ]] Instead of sharing the credentials of the AWS account root user, create individual IAM users, granting each user only the permissions they require. Follow the best practice of using the root user only to create your first IAM user. There are two types of credentials: * Root user credentials, allow full access to all resources in the AWS account. * IAM credentials, control access to AWS services and resources for users in your AWS account |
|
| Line 5: | Line 24: |
| * https://github.com/aws-samples/lambda-refarch-webapp | |
| Line 6: | Line 26: |
| * Amazon Route 53 (routes to specific places based on region) * Amazon CloudFront (deliver static content per region hosted inside S3) * Amazon Simple Storage Service (S3) * Amazon Cognito (Authentication and authorization) * Amazon API Gateway (routes requests to backend logic) * AWS Lambda (backend business logic) * AWS DynamoDB (managed DB) * AWS Identity and Access Management (IAM) - web service to control access to AWS resources == IPv6 info == * [[ https://aws.amazon.com/blogs/networking-and-content-delivery/introducing-ipv6-only-subnets-and-ec2-instances/ | introducing ipv6 only subnets and ec2 instances ]] * [[ https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-instance-addressing.html#amazon-dns | using instance addressing amazon-dns ]] * [[ https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-instance-addressing.html#working-with-ipv6-addresses | using instance addressing #working with ipv6 addresses ]] == Lambda authorizer (API gateway) == A lambda authorizer (API gateway) requires a resource based policy statement, with principalId and policy document. * [[ https://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-lambda-authorizer-output.html | API gateway lambda authorizer output ]] JSON example {{{#!highlight json { "principalId": "user|12345", "policyDocument": { "Version": "2012-10-17", "Statement": [ { "Action": "execute-api:Invoke", "Effect": "Allow", "Resource": "arn:aws:execute-api:region:account-id:api-id/stage/METHOD/path" } ] }, "context": { "stringKey": "value", "numberKey": 123, "booleanKey": true } } }}} == Add inbound rule in RDS to a lambda function == {{{#!highlight shell RDS_DB_PORT=5432 RDS_SG=$(aws rds describe-db-instances --db-instance-identifier rds-instance-id \ --query "DBInstances[0].VpcSecurityGroups[*].VpcSecurityGroupId" --output text) echo "RDS security group $RDS_SG" LAMBDA_SG=$(aws lambda get-function-configuration --function-name lambda-function-name \ --query "VpcConfig.SecurityGroupIds[*]" --output text) echo "Lambda function security group $LAMBDA_SG" aws ec2 authorize-security-group-ingress --group-id $RDS_SG \ --protocol tcp -port $RDS_DB_PORT --source-group $LAMBDA_SG }}} == Stuff == {{{#!highlight sh aws lambda list-functions aws ecs describe-task-definition -- task-definition # a task definition sets a docker image for a task/container aws apigatewayv2 get-apis aws apigatewayv2 create-api --protocol-type WEBSOCKET aws apigatewayv2 create-authorizer }}} == Lakehouse and Datawarehouse (Kimball/Inmon) == Motoserver (mock a lakehouse/datawarehouse): * S3 * Glue * Athena (DuckDB) Dimensional modelling, Athena star schema Cheap storage of a data lake (S3) High speed SQL (Athena/Redshift) ETL, extract , transform in a staging area, load in the warehouse ELT, extract raw data, load it into a data lake (S3), transform only when we need to query it Top-down Inmon method, corporate data warehouse uisng 3rd normal form (3NF) (DB normalization) * define schema/structure in AWS glue. enforce schema in all S3 buckets * use Athena federated query to get raw data into bronze raw s3 (bronze layer) * 3NF warehouse (silver layer) use Athena to transform the raw data into a normalized structure stored as a parquet files in S2. single version of the truth. Athena looks at glue catalog to turn parquet files into a 3NF table with rows and columns * Data mart (gold layer) create athena views that use the 3NF warehouse optimized for each business unit Bronze (raw) discovery and ingestion Silver (warehouse) strict 3NF, single version of truth Gold (Marts) star schema, business reporting, most similar to a cube 2010, SQL databases, MDX cubes 2026, use SQL that evolved to include "window functions" and "grouping sets" Analytical SQL (OLAP) 2026 Standard SQL (OLTP) == Deployment strategies == Blue/Green deployment (all at once) * blue is the current version * green is the new version Canary traffic shift strategy, incremental from 10% to 100% == Feature flag == New logic in code is wrapped in a if statement. Can also be implemented using class extension with beans with the same interface/contract. A router bean chooses between the older or newer code implementation instead of relying on an if statement. |
Contents
AWS
Amazon Web Services
Lambda Java samples:
User credentials
Instead of sharing the credentials of the AWS account root user, create individual IAM users, granting each user only the permissions they require.
Follow the best practice of using the root user only to create your first IAM user.
There are two types of credentials:
- Root user credentials, allow full access to all resources in the AWS account.
- IAM credentials, control access to AWS services and resources for users in your AWS account
Serverless blog web application architecture
https://s3.amazonaws.com/aws-lambda-serverless-web-refarch/RefArch_BlogApp_Serverless.png
- Amazon Route 53 (routes to specific places based on region)
Amazon CloudFront (deliver static content per region hosted inside S3)
- Amazon Simple Storage Service (S3)
- Amazon Cognito (Authentication and authorization)
- Amazon API Gateway (routes requests to backend logic)
- AWS Lambda (backend business logic)
- AWS DynamoDB (managed DB)
- AWS Identity and Access Management (IAM) - web service to control access to AWS resources
IPv6 info
Lambda authorizer (API gateway)
A lambda authorizer (API gateway) requires a resource based policy statement, with principalId and policy document.
JSON example
1 {
2 "principalId": "user|12345",
3 "policyDocument": {
4 "Version": "2012-10-17",
5 "Statement": [
6 {
7 "Action": "execute-api:Invoke",
8 "Effect": "Allow",
9 "Resource": "arn:aws:execute-api:region:account-id:api-id/stage/METHOD/path"
10 }
11 ]
12 },
13 "context": {
14 "stringKey": "value",
15 "numberKey": 123,
16 "booleanKey": true
17 }
18 }
Add inbound rule in RDS to a lambda function
1 RDS_DB_PORT=5432
2 RDS_SG=$(aws rds describe-db-instances --db-instance-identifier rds-instance-id \
3 --query "DBInstances[0].VpcSecurityGroups[*].VpcSecurityGroupId" --output text)
4 echo "RDS security group $RDS_SG"
5
6 LAMBDA_SG=$(aws lambda get-function-configuration --function-name lambda-function-name \
7 --query "VpcConfig.SecurityGroupIds[*]" --output text)
8 echo "Lambda function security group $LAMBDA_SG"
9
10 aws ec2 authorize-security-group-ingress --group-id $RDS_SG \
11 --protocol tcp -port $RDS_DB_PORT --source-group $LAMBDA_SG
Stuff
Lakehouse and Datawarehouse (Kimball/Inmon)
Motoserver (mock a lakehouse/datawarehouse):
- S3
- Glue
- Athena (DuckDB)
Dimensional modelling, Athena star schema
Cheap storage of a data lake (S3)
High speed SQL (Athena/Redshift)
ETL, extract , transform in a staging area, load in the warehouse
ELT, extract raw data, load it into a data lake (S3), transform only when we need to query it
Top-down Inmon method, corporate data warehouse uisng 3rd normal form (3NF) (DB normalization)
- define schema/structure in AWS glue. enforce schema in all S3 buckets
- use Athena federated query to get raw data into bronze raw s3 (bronze layer)
- 3NF warehouse (silver layer) use Athena to transform the raw data into a normalized structure stored as a parquet files in S2. single version of the truth. Athena looks at glue catalog to turn parquet files into a 3NF table with rows and columns
- Data mart (gold layer) create athena views that use the 3NF warehouse optimized for each business unit
Bronze (raw) discovery and ingestion Silver (warehouse) strict 3NF, single version of truth Gold (Marts) star schema, business reporting, most similar to a cube
2010, SQL databases, MDX cubes
2026, use SQL that evolved to include "window functions" and "grouping sets"
Analytical SQL (OLAP) 2026
Standard SQL (OLTP)
== Deployment strategies ==
Blue/Green deployment (all at once)
- blue is the current version
- green is the new version
Canary traffic shift strategy, incremental from 10% to 100%
Feature flag
New logic in code is wrapped in a if statement. Can also be implemented using class extension with beans with the same interface/contract. A router bean chooses between the older or newer code implementation instead of relying on an if statement.