Inserting Logs in GCP Logging by creating a NestJs Service

Inserting Logs in GCP Logging by creating a NestJs Service

Logging is a crucial aspect of application development and operation, allowing you to monitor and track the behavior and performance of your applications. Google Cloud Platform (GCP) offers a powerful logging service that enables you to store, analyze, and manage logs from various sources, including applications, services, and infrastructure. In this article, we'll explore how to insert logs into GCP Logging and discuss the advantages of using this service.

To insert logs into Google Cloud Logging, you can use the Google Cloud client library for Node.js. In this example, we will walk through the code that inserts logs into GCP Logging and explain its components.

Step 1: Setting Up the Environment

Before you can insert logs into GCP Logging, you need to set up your environment and import the required libraries. You'll need the @google-cloud/logging library and NestJS for this example. Also you need to install @google-cloud/logging by command "npm i @google-cloud/logging --save"

                    import { Logging } from '@google-cloud/logging';
                    import { Injectable } from '@nestjs/common';
                        

Step 2: Creating the CloudLoggerService Class

In the code, we create a class named CloudLoggerService. This class encapsulates the logic for inserting logs into GCP Logging.

                    @Injectable()
                    export class CloudLoggerService {
                      private logging: Logging;
                      private severityLog: string = 'LOG';
                      private resource: any = { type: 'global' };
                      private serviceName: string;
                      private logName: string;
                    
                      constructor() {
                        const projectId = 'YOUR_PROJECT_ID'; // Replace with your GCP project ID
                        this.logging = new Logging({ projectId });
                        this.serviceName = 'PROVIDE_ANY_SERVICE_NAME'; // Replace with your service name
                        this.logName = `${this.serviceName}_log`;
                      }
                    
                    }
                    
                        

In this constructor, we initialize the CloudLoggerService and configure it with the necessary information. You should replace 'YOUR_PROJECT_ID' with your GCP project ID and 'PROVIDE_ANY_SERVICE_NAME' with your service name.

Step 3: Logging a Message

The core functionality of logging a message is implemented in the log method. This method takes two arguments: message (a string) and other (an optional object of type CloudLoggerDTO). The CloudLoggerDTO class defines various metadata fields that can be included with the log message.

                    
                      log(message: string, other: CloudLoggerDTO = {}): void {
                      try {
                        const messageInfo = this.getMessageInfo(message, other);
                        const metadataLog = this.getMetaData(this.severityLog);
                        const log = this.logging.log(this.logName, this.resource);
                        const entry = this.logging.entry(metadataLog, messageInfo);
                        log.write(entry);
                        return;
                      } catch (err) {
                        console.log('Exception in log()', err);
                      }
                    }
                        

In this method, the message and additional metadata from the other object are combined to create a log message. The metadataLog object contains information about the log's severity level and the resource type, while the log object represents the log destination, and the entry contains the log data.

Explaining the Log Message

A log message typically consists of the following components:

message: This is the primary content of the log message, and it provides information about what happened or what was recorded.

service: The service name or identifier that generated the log entry. It helps you identify the source of the log.

timeStamp: The timestamp when the log entry was created. It provides information about when the event occurred.

...other: Additional metadata fields that you can include in the log message, such as userId, hostname, correlationId, and more. These fields allow you to attach relevant information to the log entry.

Here are the full codes

                    /* eslint-disable no-console */
                    import { Logging } from '@google-cloud/logging';
                    import { Injectable } from '@nestjs/common';
                    
                    class CloudLoggerDTO {
                      service?: string;
                      userId?: string;
                      hostname?: string;
                      correlationId?: string;
                      requestId?: string;
                      timestamp?: string;
                      methodName?: string;
                      callStack?: string;
                      requestMethod?: string;
                      requestURI?: string;
                    }
                    
                    @Injectable()
                    export class CloudLoggerService {
                      private logging: Logging;
                      private severityLog = 'NOTICE';
                      private severityError = 'ERROR';
                      private severityWarn = 'WARNING';
                      private resource: any = { type: 'global' };
                      private serviceName: string;
                      private logName: string;
                      private env: string;
                      private projectId: string;
                      private logSize: number;
                    
                      constructor() {
                        this.projectId = 'PROJECT_ID';
                        this.logging = new Logging({ projectId: this.projectId });
                        this.logName = 'LOG NAME';
                        this.logSize = 256;
                      }
                    
                      log(message: string, other: CloudLoggerDTO = {}): void {
                        try {
                          const messageInfo = this.getMessageInfo(message, other);
                          const metadataLog = this.getMetaData(this.severityLog);
                          const log = this.logging.log(this.logName, this.resource);
                          const entry = this.logging.entry(metadataLog, messageInfo);
                          log.write(entry);
                        } catch (err) {
                          console.log('Exception in log()', err);
                        }
                      }
                    
                      error(message: string, other: CloudLoggerDTO = {}): any {
                        try {
                          const messageInfo = this.getMessageInfo(message, other);
                          const metadataLog = this.getMetaData(this.severityError);
                          const log = this.logging.log(this.logName, this.resource);
                          const entry = this.logging.entry(metadataLog, messageInfo);
                          log.write(entry);
                          return;
                        } catch (err) {
                          console.log('Exception in info()', err);
                        }
                      }
                    
                      warn(message: string, other: CloudLoggerDTO = {}): any {
                        try {
                          const messageInfo = this.getMessageInfo(message, other);
                          const metadataLog = this.getMetaData(this.severityWarn);
                          const log = this.logging.log(this.logName, this.resource);
                          const entry = this.logging.entry(metadataLog, messageInfo);
                          log.write(entry);
                          return;
                        } catch (err) {
                          console.log('Exception in warn()', err);
                        }
                      }
                    
                      getMessageInfo(message: string, other: CloudLoggerDTO) {
                        return {
                          message,
                          service: this.serviceName,
                          env: this.env,
                          logName: this.logName,
                          projectId: this.projectId,
                          timeStamp: new Date(),
                          ...this.truncateObject(other),
                        };
                      }
                    
                      getMetaData(severity) {
                        return {
                          severity,
                          resource: this.resource,
                        };
                      }
                    
                      /* This logic will remove the error: INVALID_ARGUMENT: Log entry with size 789.3K exceeds maximum size of 256.0K */
                      truncateObject(obj) {
                      if (obj && obj.error) {
                        obj.error = this.stringifyCircularJSON(obj.error).substring(0, (Number(this.logSize) - 56) * 1024);
                      }
                      if (obj && obj.request) {
                        obj.request = this.stringifyCircularJSON(obj.request).substring(0, 30 * 1024);
                      }
                      if (obj && obj.callStack) {
                        obj.callStack = this.stringifyCircularJSON(obj.callStack).substring(0, 26 * 1024);
                      }
                      return obj;
                    }
                    
                    stringifyCircularJSON(obj: any) {
                      const seen = new Set();
                      const jsonString = JSON.stringify(obj, (key, value) => { if (typeof value === 'object' && value !== null) { if (seen.has(value)) { return '[Circular Reference]'; } seen.add(value); } return value; });
                      return jsonString;
                    }
                    }
                        

Advantages of Inserting Logs in GCP Logging

Inserting logs in GCP Logging offers several advantages:

1. Centralized Log Management: GCP Logging provides a centralized platform to store and manage logs from various sources, making it easy to monitor and troubleshoot issues across your entire infrastructure.

2. Scalability: GCP Logging is highly scalable and can handle large volumes of logs, allowing you to log events and data without worrying about resource limitations.

3. Powerful Search and Filtering: With GCP Logging, you can search, filter, and analyze logs using a powerful query language. This makes it easy to find specific log entries and gain insights into your application's behavior.

4. Integration with GCP Services: GCP Logging seamlessly integrates with other Google Cloud services, such as Cloud Monitoring and Cloud Trace, enabling you to correlate logs with metrics and traces for comprehensive observability.

5. Alerting and Notifications: You can set up alerts and notifications based on specific log events or conditions, ensuring that you are alerted to critical issues in real-time.

6. Long-Term Storage and Archiving: GCP Logging allows you to retain logs for extended periods, ensuring that you can access historical log data when needed for compliance or analysis.

---

Conclusion:

Google Cloud Platform (GCP) Logging provides a robust and feature-rich solution for managing and analyzing log data from your applications and services. By following the steps outlined in this article, you can easily insert logs into GCP Logging and take advantage of its powerful features for log management, monitoring, and troubleshooting. Leveraging GCP Logging can significantly enhance your application's reliability and operational efficiency, making it a valuable tool for modern cloud-based applications.

Related Keywords:

Inserting Logs in GCP Logging

Advantages of Inserting Logs in GCP Logging

Sending GCP Logs

Google Cloud Logging Integration

Google Cloud Logging

Inserting logs in gcp logging