Merge pull request #71 from NHSDigital/feature/CCM-9868_splunk

CCM-9868: Adding cloudwatch log subscription to lambda module

Author: aidenvaines-cgi

infrastructure/modules/lambda/README.md

@@ -19,6 +19,7 @@
 | <a name="input_enable_dlq_and_notifications"></a> [enable\_dlq\_and\_notifications](#input\_enable\_dlq\_and\_notifications) | Create an SQS Queue and on-failure destination to be used as the Lambda's Dead Letter Queue and notifications | `bool` | `false` | no |
 | <a name="input_enable_lambda_insights"></a> [enable\_lambda\_insights](#input\_enable\_lambda\_insights) | Enable the lambda insights layer, this must be disabled for lambda@edge usage | `bool` | `true` | no |
 | <a name="input_environment"></a> [environment](#input\_environment) | The name of the tfscaffold environment | `string` | n/a | yes |
+| <a name="input_filter_pattern"></a> [filter\_pattern](#input\_filter\_pattern) | Filter pattern to use for the log subscription filter | `string` | `""` | no |
 | <a name="input_force_lambda_code_deploy"></a> [force\_lambda\_code\_deploy](#input\_force\_lambda\_code\_deploy) | If the lambda package in s3 has the same commit id tag as the terraform build branch, the lambda will not update automatically. Set to True if making changes to Lambda code from on the same commit for example during development | `bool` | `false` | no |
 | <a name="input_function_code_base_path"></a> [function\_code\_base\_path](#input\_function\_code\_base\_path) | The path to the sourcecode directories needed for this lambda | `string` | `"./"` | no |
 | <a name="input_function_code_dir"></a> [function\_code\_dir](#input\_function\_code\_dir) | The path to the sourcecode directories needed for this lambda | `string` | n/a | yes |
@@ -34,14 +35,17 @@
 | <a name="input_lambda_dlq_message_retention_seconds"></a> [lambda\_dlq\_message\_retention\_seconds](#input\_lambda\_dlq\_message\_retention\_seconds) | KMS Key ARN to be used for SNS Topic for on-failure Lambda invocation records | `number` | `86400` | no |
 | <a name="input_lambda_env_vars"></a> [lambda\_env\_vars](#input\_lambda\_env\_vars) | Lambda environment parameters map | `map(string)` | `{}` | no |
 | <a name="input_layers"></a> [layers](#input\_layers) | Lambda layer arns to include | `list(any)` | `[]` | no |
+| <a name="input_log_destination_arn"></a> [log\_destination\_arn](#input\_log\_destination\_arn) | Destination ARN to use for the log subscription filter | `string` | `""` | no |
 | <a name="input_log_level"></a> [log\_level](#input\_log\_level) | The log level to be used in lambda functions within the component. Any log with a lower severity than the configured value will not be logged: https://docs.python.org/3/library/logging.html#levels | `string` | `"INFO"` | no |
 | <a name="input_log_retention_in_days"></a> [log\_retention\_in\_days](#input\_log\_retention\_in\_days) | The retention period in days for the Cloudwatch Logs events generated by the lambda function | `number` | n/a | yes |
 | <a name="input_log_subscription_lambda_create_permission"></a> [log\_subscription\_lambda\_create\_permission](#input\_log\_subscription\_lambda\_create\_permission) | Whether to create a permission for the log forwarder. Set to false if using a generic one. | `bool` | `true` | no |
+| <a name="input_log_subscription_role_arn"></a> [log\_subscription\_role\_arn](#input\_log\_subscription\_role\_arn) | The ARN of the IAM role to use for the log subscription filter | `string` | `""` | no |
 | <a name="input_memory"></a> [memory](#input\_memory) | The amount of memory to apply to the created Lambda | `number` | n/a | yes |
 | <a name="input_project"></a> [project](#input\_project) | The name of the tfscaffold project | `string` | n/a | yes |
 | <a name="input_region"></a> [region](#input\_region) | The AWS Region | `string` | n/a | yes |
 | <a name="input_runtime"></a> [runtime](#input\_runtime) | The runtime to use for the lambda function | `string` | n/a | yes |
 | <a name="input_schedule"></a> [schedule](#input\_schedule) | The fully qualified Cloudwatch Events schedule for when to run the lambda function, e.g. rate(1 day) or a cron() expression. Default disables all events resources | `string` | `""` | no |
+| <a name="input_send_to_firehose"></a> [send\_to\_firehose](#input\_send\_to\_firehose) | Enable sending logs to firehose | `bool` | `false` | no |
 | <a name="input_sns_destination"></a> [sns\_destination](#input\_sns\_destination) | SNS Topic ARN to be used for on-failure Lambda invocation records | `string` | `null` | no |
 | <a name="input_sns_destination_kms_key"></a> [sns\_destination\_kms\_key](#input\_sns\_destination\_kms\_key) | KMS Key ARN to be used for SNS Topic for on-failure Lambda invocation records | `string` | `null` | no |
 | <a name="input_system_log_level"></a> [system\_log\_level](#input\_system\_log\_level) | The detail level of the Lambda platform event logs sent to CloudWatch | `string` | `"WARN"` | no |

infrastructure/modules/lambda/cloudwatch_log_subscription_filter_firehose.tf

@@ -0,0 +1,8 @@
+resource "aws_cloudwatch_log_subscription_filter" "firehose" {
+  count           = var.send_to_firehose ? 1 : 0 # Keeping this optional for now as don't want to break all lambdas using this, can make this mandatory later
+  name            = replace(aws_cloudwatch_log_group.main.name, "/", "-")
+  log_group_name  = aws_cloudwatch_log_group.main.name
+  filter_pattern  = var.filter_pattern
+  destination_arn = var.log_destination_arn
+  role_arn        = var.log_subscription_role_arn
+}

infrastructure/modules/lambda/variables.tf

@@ -212,3 +212,27 @@ variable "lambda_at_edge" {
   description = "Enable the lambda insights layer, this must be disabled for lambda@edge usage"
   default     = false
 }
+
+variable "send_to_firehose" {
+  type        = bool
+  description = "Enable sending logs to firehose"
+  default     = false
+}
+
+variable "filter_pattern" {
+  type        = string
+  description = "Filter pattern to use for the log subscription filter"
+  default     = ""
+}
+
+variable "log_destination_arn" {
+  type        = string
+  description = "Destination ARN to use for the log subscription filter"
+  default     = ""
+}
+
+variable "log_subscription_role_arn" {
+  type        = string
+  description = "The ARN of the IAM role to use for the log subscription filter"
+  default     = ""
+}