Skip to content

Commit 3d344f1

Browse files
authored
chore(docs): update import log README (GoogleCloudPlatform#10991)
## Description add instructions about importing logs older than 30 days to the sample's README file.
1 parent 5bb78f4 commit 3d344f1

File tree

1 file changed

+33
-0
lines changed

1 file changed

+33
-0
lines changed

logging/import-logs/README.md

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -91,3 +91,36 @@ nox -s lint
9191
nox -s py-3.8
9292
nox -s py-3.11
9393
```
94+
95+
## Importing log entries with timestamps older than 30 days
96+
97+
The import logs implementation does not currently support logs with the timestamp older than 30 days
98+
because the incoming log entries must not be older than default retention period (see [documentation][retention]).
99+
To prevent such logs from being ignored the implementation returns errors if the timestamp is older than 29 days.
100+
To import the older logs you have to modified the [current] code by performing the following modifications:
101+
102+
* remove the fencing condition [lines][code1]:
103+
104+
<https://github.com/GoogleCloudPlatform/python-docs-samples/blob/95dd4f53ff96470a1f842d3134d56b017a85ac27/logging/import-logs/main.py#L91-L93>
105+
106+
* in [`import_logs`][code2] add the following block after the call to [`_patch_reserved_log_ids`][code3]:
107+
108+
```python
109+
log.labels['original_timestamp'] = log.timestamp
110+
log.timestamp = None
111+
```
112+
113+
to keep the original timestamp as a user metadata labeled `original_timestamp` and to ingest the log entry using _current_ timestamp.
114+
115+
> [!IMPORTANT]
116+
>
117+
> 1. After this change the logs should be queried using the `timestamp` field.
118+
> 1. If the same log entry is imported multiple times, the query response may include more than one line.
119+
120+
After applying the changes, [build](#build) a custom container image and use it when creating an import job.
121+
122+
[retention]: https://cloud.google.com/logging/docs/reference/v2/rest/v2/LogEntry#FIELDS.timestamp
123+
[current]: https://github.com/GoogleCloudPlatform/python-docs-samples/blob/e2709a218072c86ec1a9b9101db45057ebfdbff0/logging/import-logs/main.py
124+
[code1]: https://github.com/GoogleCloudPlatform/python-docs-samples/blob/95dd4f53ff96470a1f842d3134d56b017a85ac27/logging/import-logs/main.py#L91-L93
125+
[code2]: https://github.com/GoogleCloudPlatform/python-docs-samples/blob/95dd4f53ff96470a1f842d3134d56b017a85ac27/logging/import-logs/main.py#L196
126+
[code3]: https://github.com/GoogleCloudPlatform/python-docs-samples/blob/95dd4f53ff96470a1f842d3134d56b017a85ac27/logging/import-logs/main.py#L206

0 commit comments

Comments
 (0)