File tree Expand file tree Collapse file tree 1 file changed +33
-0
lines changed Expand file tree Collapse file tree 1 file changed +33
-0
lines changed Original file line number Diff line number Diff line change @@ -91,3 +91,36 @@ nox -s lint
9191nox -s py-3.8
9292nox -s py-3.11
9393```
94+
95+ ## Importing log entries with timestamps older than 30 days
96+
97+ The import logs implementation does not currently support logs with the timestamp older than 30 days
98+ because the incoming log entries must not be older than default retention period (see [ documentation] [ retention ] ).
99+ To prevent such logs from being ignored the implementation returns errors if the timestamp is older than 29 days.
100+ To import the older logs you have to modified the [ current] code by performing the following modifications:
101+
102+ * remove the fencing condition [ lines] [ code1 ] :
103+
104+ < https://github.com/GoogleCloudPlatform/python-docs-samples/blob/95dd4f53ff96470a1f842d3134d56b017a85ac27/logging/import-logs/main.py#L91-L93 >
105+
106+ * in [ ` import_logs ` ] [ code2 ] add the following block after the call to [ ` _patch_reserved_log_ids ` ] [ code3 ] :
107+
108+ ``` python
109+ log.labels[' original_timestamp' ] = log.timestamp
110+ log.timestamp = None
111+ ```
112+
113+ to keep the original timestamp as a user metadata labeled ` original_timestamp ` and to ingest the log entry using _ current_ timestamp.
114+
115+ > [ !IMPORTANT]
116+ >
117+ > 1 . After this change the logs should be queried using the ` timestamp ` field.
118+ > 1 . If the same log entry is imported multiple times, the query response may include more than one line.
119+
120+ After applying the changes, [ build] ( #build ) a custom container image and use it when creating an import job.
121+
122+ [ retention ] : https://cloud.google.com/logging/docs/reference/v2/rest/v2/LogEntry#FIELDS.timestamp
123+ [ current ] : https://github.com/GoogleCloudPlatform/python-docs-samples/blob/e2709a218072c86ec1a9b9101db45057ebfdbff0/logging/import-logs/main.py
124+ [ code1 ] : https://github.com/GoogleCloudPlatform/python-docs-samples/blob/95dd4f53ff96470a1f842d3134d56b017a85ac27/logging/import-logs/main.py#L91-L93
125+ [ code2 ] : https://github.com/GoogleCloudPlatform/python-docs-samples/blob/95dd4f53ff96470a1f842d3134d56b017a85ac27/logging/import-logs/main.py#L196
126+ [ code3 ] : https://github.com/GoogleCloudPlatform/python-docs-samples/blob/95dd4f53ff96470a1f842d3134d56b017a85ac27/logging/import-logs/main.py#L206
You can’t perform that action at this time.
0 commit comments