- Notifications
You must be signed in to change notification settings - Fork 101
feat: Add support for multiplexed sessions #1381
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
.github/workflows/integration-tests-against-emulator-with-multiplexed-session.yaml Fixed Show fixed Hide fixed
f04b0e1 to e8f813c Compare e8f813c to 69f28c8 Compare fb74975 to 8899249 Compare 53f0cc5 to 6dc6b4b Compare ebdceba to bbbc41e Compare .github/workflows/integration-tests-against-emulator-with-multiplexed-session.yaml Dismissed Show dismissed Hide dismissed
841408a to 146f11a Compare e83298f to aa0e2a5 Compare 4c8a19b to d7c54cd Compare graph TD A[Client Request] --> B{Multiplexed Sessions Enabled?} B -->|No| C[Regular Session Behavior] B -->|Yes| D[Multiplexed Session Behavior] %% Regular Session Path C --> C1[Read-Only Operations] C --> C2[Partitioned Operations] C --> C3[Read-Write Operations] C1 --> C1A[Get Session from Pool] C1A --> C1B[Create/Use Regular Session] C1B --> C1C[Execute Read Operation] C1C --> C1D[Return Session to Pool] C2 --> C2A[BatchSnapshot Creation] C2A --> C2B[Get Session from Pool<br/>TransactionType.PARTITIONED] C2B --> C2C[Create Regular Session] C2C --> C2D[Generate Partitions] C2D --> C2E[Each Partition Uses<br/>SAME Session] C2E --> C2F[Process All Partitions<br/>Sequentially] C3 --> C3A[Get Session from Pool] C3A --> C3B[Create/Use Regular Session] C3B --> C3C[Execute Transaction] C3C --> C3D[Return Session to Pool] %% Multiplexed Session Path D --> D1[Read-Only Operations] D --> D2[Partitioned Operations] D --> D3[Read-Write Operations] D1 --> D1A[Get Multiplexed Session] D1A --> D1B{Multiplexed Session Exists?} D1B -->|No| D1C[Create Multiplexed Session] D1B -->|Yes| D1D[Reuse Existing Session] D1C --> D1E[Start Maintenance Thread] D1E --> D1F[Execute Read Operation] D1D --> D1F D1F --> D1G[Session Remains Active<br/>No Return Needed] D2 --> D2A[BatchSnapshot Creation] D2A --> D2B[Get Multiplexed Session<br/>TransactionType.PARTITIONED] D2B --> D2C{Multiplexed Session Exists?} D2C -->|No| D2D[Create Multiplexed Session] D2C -->|Yes| D2E[Reuse Existing Session] D2D --> D2F[Generate Partitions] D2E --> D2F D2F --> D2G[Each Partition Uses<br/>SAME Multiplexed Session] D2G --> D2H[Process All Partitions<br/>Concurrently Possible] D3 --> D3A[Fallback to Regular Session] D3A --> D3B[Get Session from Pool] D3B --> D3C[Create/Use Regular Session] D3C --> D3D[Execute Transaction] D3D --> D3E[Return Session to Pool] %% Environment Variables E[Environment Variables] --> E1[GOOGLE_CLOUD_SPANNER_MULTIPLEXED_SESSIONS=true] E --> E2[GOOGLE_CLOUD_SPANNER_MULTIPLEXED_SESSIONS_PARTITIONED_OPS=true] E1 --> B E2 --> D2 %% Key Differences F[Key Differences] --> F1[Session Reuse] F --> F2[Concurrency] F --> F3[Resource Usage] F1 --> F1A[Regular: New session per operation] F1 --> F1B[Multiplexed: Same session shared] F2 --> F2A[Regular: Sequential partition processing] F2 --> F2B[Multiplexed: Concurrent partition processing] F3 --> F3A[Regular: Multiple sessions = more resources] F3 --> F3B[Multiplexed: Single session = fewer resources] %% Styling classDef regularPath fill:#ffcccc,stroke:#ff0000,stroke-width:2px classDef multiplexedPath fill:#ccffcc,stroke:#00ff00,stroke-width:2px classDef envVar fill:#ffffcc,stroke:#ffaa00,stroke-width:2px classDef keyDiff fill:#ccccff,stroke:#0000ff,stroke-width:2px class C,C1,C2,C3,C1A,C1B,C1C,C1D,C2A,C2B,C2C,C2D,C2E,C2F,C3A,C3B,C3C,C3D regularPath class D,D1,D2,D3,D1A,D1B,D1C,D1D,D1E,D1F,D1G,D2A,D2B,D2C,D2D,D2E,D2F,D2G,D2H multiplexedPath class E,E1,E2 envVar class F,F1,F2,F3,F1A,F1B,F2A,F2B,F3A,F3B keyDiff |
| rows1 = list(snap1.read(sd.TABLE, sd.COLUMNS, sd.ALL)) | ||
| rows2 = list(snap2.read(sd.TABLE, sd.COLUMNS, sd.ALL)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Tests seem to be failing here because the snapshot's session has does not exist. This means that Session.create() has not been called. Not sure exactly why that wouldn't be the case, but tough to find out without being able to run locally.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Issue was some tests were calling session.delete which prematurely deletes the multiplex session and hence "Session Not Found" error, Using Emulator calling session.delete() does not actually delete it from Emulator so issue does not occur using Emulator
d7c54cd to 9f2046a Compare 4e57db2 to 16954a6 Compare 16954a6 to 94b88dd Compare
Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:
Fixes #<issue_number_goes_here> 🦕