Commit c40b09b
committed
[#27723] YSQL: Forward startup pkt from conn mgr to auth backend
Summary:
Connection Manager intercepts the startup packet from external client to store the GUC settings it contains, so they can be replayed on any transactional backend it later attaches the client to. This has a few problems, the primary one being that the startup packet may contain variables that cannot be set through `SET` statements (the mechanism connection manager uses for replaying). Additionally, an incorrect GUC setting like `geqo = "xyz"` (setting a string value for a bool GUC) might be there in the startup packet and so the connection should be closed at startup time itself.
With this revision, we forward the startup packet to the backend we spawn for authentication. Then, we save the settings from the returned `YbParameterStatus` packets (type 'r') instead of directly storing settings from the startup packet. This has many benefits:
- This handles GUC variables that have been set from `PGC_S_OVERRIDE` source in a new backend. The startup packet setting don't affect these variables (as it has a lower priority) and so that setting shouldn't be replayed
- Since we now only store GUC names from `ParameterStatus` packets, this eliminates the need for any case matching on connection manager side. This is because Postgres always sends the GUC names in the same casing, as specified in the GUC definition
- Due to this, we can remove the lowercasing code for auth backend. It has to stay for auth passthrough till we do something similar for it.
- Also added a Java test for this to verify that case insensitivity of GUC settings remains
- Handle precedence of GUC settings in startup packet. Now that we don't have to parse the startup packet, we can just rely on Postgres determining the correct precedence between GUC setting through "options" field or directly.
- Added a Java test for this
NOTE: Regarding auth passthrough -- This diff doesn't impact auth passthrough functionality at all, we only do the new things for auth backend. We continue lowercasing variables and not forwarding startup packet settings for authentication passthrough.
Some code changes which require explanation:
- `kiwi_vars_init` now takes an additional parameter: It used to always add `compute_query_id` and `pg_hint_table.enable_hint_table` variables to the vars, now it does so conditionally. This is because we're using the same struct to save startup settings that are forwarded to auth backend and we don't want these variable to also be sent.
- We parse the options sent in startup packet and lowercase GUC variable names only in the case of auth passthrough now. Methods are added in `instance.c` to determine when to parse options & lowercase variable names
- We also populate these flag bits in YbParameterStatus now:
- `YB_PARAM_STATUS_CONTEXT_BACKEND` : Means that a variable with context `PGC_{SU_}BACKEND` has been set from source `PGC_S_CLIENT` (startup packet).
- `YB_PARAM_STATUS_USERSET_SOURCE_STARTUP` : Variable with context `PGC_SUSET` / `PGC_USERSET` has been set from source `PGC_S_CLIENT` (startup packet)
- `YB_PARAM_STATUS_USERSET_SOURCE_SESSION` : Variable with context `PGC_SUSET` / `PGC_USERSET` has been set from source `PGC_S_SESSION` (`SET` statement)
- These flags are used in authentication flow so that we only save the GUC settings that have been caused by the startup packet. Additionally, we directly send ParameterStatus back to client for packets with YB_PARAM_STATUS_REPORT_ENABLED bit set.
Test Plan:
The following tests have been added in this diff:
```
./yb_build.sh release --enable-ysql-conn-mgr-test --java-test org.yb.ysqlconnmgr.TestSessionParameters#testStartupParameterPrecedence
./yb_build.sh release --enable-ysql-conn-mgr-test --java-test org.yb.ysqlconnmgr.TestSessionParameters#testStartupParameterCaseInsensitivity
./yb_build.sh release --enable-ysql-conn-mgr-test --java-test org.yb.ysqlconnmgr.TestSessionParameters#testInvalidStartupParameter
./yb_build.sh release --enable-ysql-conn-mgr-test --java-test org.yb.ysqlconnmgr.TestSessionParameters#testSettingOverrideVariableInStartupPacket
```
Jenkins: all tests
Reviewers: skumar, mkumar, asrinivasan, vikram.damle
Reviewed By: skumar
Subscribers: yql
Tags: #jenkins-ready
Differential Revision: https://phorge.dev.yugabyte.com/D449461 parent a327e4f commit c40b09b
File tree
18 files changed
+564
-263
lines changed- java/yb-ysql-conn-mgr/src/test/java/org/yb/ysqlconnmgr
- src
- odyssey
- sources
- third_party/kiwi/kiwi
- postgres/src
- backend/utils/misc
- include/common
18 files changed
+564
-263
lines changedLines changed: 160 additions & 30 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
15 | 15 | | |
16 | 16 | | |
17 | 17 | | |
| 18 | + | |
18 | 19 | | |
| 20 | + | |
19 | 21 | | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
20 | 25 | | |
21 | 26 | | |
22 | 27 | | |
| |||
25 | 30 | | |
26 | 31 | | |
27 | 32 | | |
28 | | - | |
| 33 | + | |
29 | 34 | | |
30 | 35 | | |
| 36 | + | |
| 37 | + | |
31 | 38 | | |
32 | 39 | | |
33 | 40 | | |
| |||
118 | 125 | | |
119 | 126 | | |
120 | 127 | | |
| 128 | + | |
121 | 129 | | |
122 | | - | |
| 130 | + | |
123 | 131 | | |
124 | 132 | | |
125 | 133 | | |
| |||
546 | 554 | | |
547 | 555 | | |
548 | 556 | | |
549 | | - | |
550 | | - | |
| 557 | + | |
| 558 | + | |
551 | 559 | | |
552 | 560 | | |
553 | 561 | | |
554 | 562 | | |
555 | 563 | | |
556 | | - | |
| 564 | + | |
557 | 565 | | |
558 | 566 | | |
559 | 567 | | |
| |||
566 | 574 | | |
567 | 575 | | |
568 | 576 | | |
569 | | - | |
570 | | - | |
571 | | - | |
| 577 | + | |
| 578 | + | |
| 579 | + | |
| 580 | + | |
| 581 | + | |
572 | 582 | | |
573 | 583 | | |
574 | 584 | | |
575 | 585 | | |
576 | 586 | | |
577 | | - | |
| 587 | + | |
578 | 588 | | |
579 | 589 | | |
580 | 590 | | |
581 | | - | |
582 | | - | |
583 | | - | |
| 591 | + | |
| 592 | + | |
| 593 | + | |
| 594 | + | |
| 595 | + | |
584 | 596 | | |
585 | 597 | | |
586 | 598 | | |
587 | 599 | | |
588 | 600 | | |
589 | | - | |
| 601 | + | |
590 | 602 | | |
591 | 603 | | |
| 604 | + | |
592 | 605 | | |
| 606 | + | |
| 607 | + | |
| 608 | + | |
| 609 | + | |
| 610 | + | |
| 611 | + | |
| 612 | + | |
| 613 | + | |
| 614 | + | |
| 615 | + | |
| 616 | + | |
| 617 | + | |
| 618 | + | |
| 619 | + | |
| 620 | + | |
| 621 | + | |
| 622 | + | |
| 623 | + | |
| 624 | + | |
| 625 | + | |
| 626 | + | |
| 627 | + | |
| 628 | + | |
| 629 | + | |
| 630 | + | |
| 631 | + | |
| 632 | + | |
| 633 | + | |
| 634 | + | |
| 635 | + | |
| 636 | + | |
| 637 | + | |
| 638 | + | |
| 639 | + | |
| 640 | + | |
| 641 | + | |
| 642 | + | |
| 643 | + | |
| 644 | + | |
| 645 | + | |
| 646 | + | |
| 647 | + | |
| 648 | + | |
| 649 | + | |
| 650 | + | |
| 651 | + | |
| 652 | + | |
| 653 | + | |
| 654 | + | |
| 655 | + | |
| 656 | + | |
| 657 | + | |
| 658 | + | |
| 659 | + | |
| 660 | + | |
| 661 | + | |
| 662 | + | |
| 663 | + | |
| 664 | + | |
| 665 | + | |
| 666 | + | |
| 667 | + | |
| 668 | + | |
| 669 | + | |
| 670 | + | |
| 671 | + | |
| 672 | + | |
| 673 | + | |
| 674 | + | |
| 675 | + | |
| 676 | + | |
| 677 | + | |
| 678 | + | |
| 679 | + | |
| 680 | + | |
| 681 | + | |
| 682 | + | |
| 683 | + | |
| 684 | + | |
| 685 | + | |
| 686 | + | |
| 687 | + | |
| 688 | + | |
| 689 | + | |
| 690 | + | |
| 691 | + | |
| 692 | + | |
| 693 | + | |
| 694 | + | |
| 695 | + | |
| 696 | + | |
| 697 | + | |
| 698 | + | |
| 699 | + | |
| 700 | + | |
| 701 | + | |
| 702 | + | |
| 703 | + | |
| 704 | + | |
| 705 | + | |
593 | 706 | | |
594 | | - | |
595 | | - | |
| 707 | + | |
| 708 | + | |
| 709 | + | |
596 | 710 | | |
597 | | - | |
598 | | - | |
599 | | - | |
| 711 | + | |
| 712 | + | |
600 | 713 | | |
601 | | - | |
602 | | - | |
603 | | - | |
604 | | - | |
| 714 | + | |
| 715 | + | |
| 716 | + | |
| 717 | + | |
| 718 | + | |
| 719 | + | |
| 720 | + | |
| 721 | + | |
| 722 | + | |
605 | 723 | | |
606 | 724 | | |
| 725 | + | |
| 726 | + | |
| 727 | + | |
| 728 | + | |
| 729 | + | |
| 730 | + | |
| 731 | + | |
| 732 | + | |
| 733 | + | |
| 734 | + | |
| 735 | + | |
| 736 | + | |
| 737 | + | |
607 | 738 | | |
608 | 739 | | |
609 | | - | |
| 740 | + | |
| 741 | + | |
610 | 742 | | |
611 | | - | |
612 | | - | |
613 | | - | |
614 | 743 | | |
615 | | - | |
616 | | - | |
617 | | - | |
618 | | - | |
| 744 | + | |
| 745 | + | |
| 746 | + | |
| 747 | + | |
| 748 | + | |
619 | 749 | | |
620 | 750 | | |
621 | 751 | | |
0 commit comments