Skip to content

Commit c40b09b

Browse files
committed
[#27723] YSQL: Forward startup pkt from conn mgr to auth backend
Summary: Connection Manager intercepts the startup packet from external client to store the GUC settings it contains, so they can be replayed on any transactional backend it later attaches the client to. This has a few problems, the primary one being that the startup packet may contain variables that cannot be set through `SET` statements (the mechanism connection manager uses for replaying). Additionally, an incorrect GUC setting like `geqo = "xyz"` (setting a string value for a bool GUC) might be there in the startup packet and so the connection should be closed at startup time itself. With this revision, we forward the startup packet to the backend we spawn for authentication. Then, we save the settings from the returned `YbParameterStatus` packets (type 'r') instead of directly storing settings from the startup packet. This has many benefits: - This handles GUC variables that have been set from `PGC_S_OVERRIDE` source in a new backend. The startup packet setting don't affect these variables (as it has a lower priority) and so that setting shouldn't be replayed - Since we now only store GUC names from `ParameterStatus` packets, this eliminates the need for any case matching on connection manager side. This is because Postgres always sends the GUC names in the same casing, as specified in the GUC definition - Due to this, we can remove the lowercasing code for auth backend. It has to stay for auth passthrough till we do something similar for it. - Also added a Java test for this to verify that case insensitivity of GUC settings remains - Handle precedence of GUC settings in startup packet. Now that we don't have to parse the startup packet, we can just rely on Postgres determining the correct precedence between GUC setting through "options" field or directly. - Added a Java test for this NOTE: Regarding auth passthrough -- This diff doesn't impact auth passthrough functionality at all, we only do the new things for auth backend. We continue lowercasing variables and not forwarding startup packet settings for authentication passthrough. Some code changes which require explanation: - `kiwi_vars_init` now takes an additional parameter: It used to always add `compute_query_id` and `pg_hint_table.enable_hint_table` variables to the vars, now it does so conditionally. This is because we're using the same struct to save startup settings that are forwarded to auth backend and we don't want these variable to also be sent. - We parse the options sent in startup packet and lowercase GUC variable names only in the case of auth passthrough now. Methods are added in `instance.c` to determine when to parse options & lowercase variable names - We also populate these flag bits in YbParameterStatus now: - `YB_PARAM_STATUS_CONTEXT_BACKEND` : Means that a variable with context `PGC_{SU_}BACKEND` has been set from source `PGC_S_CLIENT` (startup packet). - `YB_PARAM_STATUS_USERSET_SOURCE_STARTUP` : Variable with context `PGC_SUSET` / `PGC_USERSET` has been set from source `PGC_S_CLIENT` (startup packet) - `YB_PARAM_STATUS_USERSET_SOURCE_SESSION` : Variable with context `PGC_SUSET` / `PGC_USERSET` has been set from source `PGC_S_SESSION` (`SET` statement) - These flags are used in authentication flow so that we only save the GUC settings that have been caused by the startup packet. Additionally, we directly send ParameterStatus back to client for packets with YB_PARAM_STATUS_REPORT_ENABLED bit set. Test Plan: The following tests have been added in this diff: ``` ./yb_build.sh release --enable-ysql-conn-mgr-test --java-test org.yb.ysqlconnmgr.TestSessionParameters#testStartupParameterPrecedence ./yb_build.sh release --enable-ysql-conn-mgr-test --java-test org.yb.ysqlconnmgr.TestSessionParameters#testStartupParameterCaseInsensitivity ./yb_build.sh release --enable-ysql-conn-mgr-test --java-test org.yb.ysqlconnmgr.TestSessionParameters#testInvalidStartupParameter ./yb_build.sh release --enable-ysql-conn-mgr-test --java-test org.yb.ysqlconnmgr.TestSessionParameters#testSettingOverrideVariableInStartupPacket ``` Jenkins: all tests Reviewers: skumar, mkumar, asrinivasan, vikram.damle Reviewed By: skumar Subscribers: yql Tags: #jenkins-ready Differential Revision: https://phorge.dev.yugabyte.com/D44946
1 parent a327e4f commit c40b09b

File tree

18 files changed

+564
-263
lines changed

18 files changed

+564
-263
lines changed

java/yb-ysql-conn-mgr/src/test/java/org/yb/ysqlconnmgr/TestSessionParameters.java

Lines changed: 160 additions & 30 deletions
Original file line numberDiff line numberDiff line change
@@ -15,8 +15,13 @@
1515

1616
import static org.yb.AssertionWrappers.assertTrue;
1717
import static org.yb.AssertionWrappers.assertFalse;
18+
import static org.yb.AssertionWrappers.assertEquals;
1819
import static org.yb.AssertionWrappers.fail;
20+
import static org.yb.AssertionWrappers.assertThrows;
1921

22+
import java.io.File;
23+
import java.io.IOException;
24+
import java.net.InetSocketAddress;
2025
import java.sql.Connection;
2126
import java.sql.ResultSet;
2227
import java.sql.Statement;
@@ -25,9 +30,11 @@
2530
import java.util.Set;
2631
import java.util.HashSet;
2732
import java.util.Arrays;
28-
33+
import java.util.List;
2934
import org.junit.Test;
3035
import org.junit.runner.RunWith;
36+
import org.yb.client.TestUtils;
37+
import org.yb.util.ProcessUtil;
3138
import org.yb.minicluster.MiniYBClusterBuilder;
3239
import org.yb.pgsql.ConnectionEndpoint;
3340
import com.yugabyte.PGConnection;
@@ -118,8 +125,9 @@ private static enum ExceptionType {
118125
new ExceptionType[] {}),
119126
new SessionParameter("search_path", "\"$user\", public", "test",
120127
new ExceptionType[] {}),
128+
// JDBC sends TimeZone in the startup packet, and it can't be overridden using startup options
121129
new SessionParameter("TimeZone", "UTC", "EST",
122-
new ExceptionType[] {}),
130+
new ExceptionType[] { ExceptionType.INVALID_STARTUP }),
123131
new SessionParameter("default_transaction_isolation", "read committed", "serializable",
124132
new ExceptionType[] { ExceptionType.YB_CACHE_MAPPING }),
125133
new SessionParameter("transaction_isolation", "read committed",
@@ -546,14 +554,14 @@ public void testStartupParameters() throws Exception {
546554
String parameterName = sp.parameterName;
547555
String expectedValue = sp.expectedValue;
548556

549-
// (GH#22248) (rbarigidad) Startup parameters with spaces are not parsed
550-
// correctly, they are ignored for now as well.
557+
String valueWithEscapedSpaces = expectedValue.replace(" ", "\\ ");
558+
551559
if (sp.exceptionSet.contains(ExceptionType.INVALID_STARTUP))
552560
continue;
553561

554562
try (Connection conn = getConnectionBuilder()
555563
.withConnectionEndpoint(ConnectionEndpoint.YSQL_CONN_MGR)
556-
.withOptions(String.format("-c %s=%s", parameterName, expectedValue))
564+
.withOptions(String.format("-c %s=%s", parameterName, valueWithEscapedSpaces))
557565
.connect();
558566
Statement stmt = conn.createStatement()) {
559567
String fetchedValue = fetchParameterValue(stmt, sp);
@@ -566,56 +574,178 @@ public void testStartupParameters() throws Exception {
566574

567575
@Test
568576
public void testTimeZoneSetting() throws Exception {
569-
try (Connection conn =
570-
getConnectionBuilder().withConnectionEndpoint(ConnectionEndpoint.YSQL_CONN_MGR)
571-
.withOptions("-c TimeZone=Asia/Kolkata").connect()) {
577+
try (Connection conn = getConnectionBuilder()
578+
.withConnectionEndpoint(ConnectionEndpoint.YSQL_CONN_MGR)
579+
.connect();
580+
Statement stmt = conn.createStatement()) {
581+
stmt.execute("SET TimeZone = 'Asia/Kolkata'");
572582
PGConnection pgConn = (PGConnection) conn;
573583
Map<String, String> params = new HashMap<>(pgConn.getParameterStatuses());
574584

575585
// Get the timezone directly
576586
String backendTimeZone = params.get("TimeZone");
577-
assertTrue("Backend timezone should be set correctly from startup packet",
587+
assertTrue("Backend timezone should be set correctly from SET command",
578588
backendTimeZone.equals("Asia/Kolkata"));
579589
}
580590

581-
try (Connection conn =
582-
getConnectionBuilder().withConnectionEndpoint(ConnectionEndpoint.YSQL_CONN_MGR)
583-
.withOptions("-c timezone=Asia/Kolkata").connect()) {
591+
try (Connection conn = getConnectionBuilder()
592+
.withConnectionEndpoint(ConnectionEndpoint.YSQL_CONN_MGR)
593+
.connect();
594+
Statement stmt = conn.createStatement()) {
595+
stmt.execute("SET timezone = 'Asia/Kolkata'");
584596
PGConnection pgConn = (PGConnection) conn;
585597
Map<String, String> params = new HashMap<>(pgConn.getParameterStatuses());
586598

587599
// Get the timezone directly
588600
String backendTimeZone = params.get("TimeZone");
589-
assertTrue("Backend timezone should be set correctly from startup packet",
601+
assertTrue("Backend timezone should be set correctly from SET command",
590602
backendTimeZone.equals("Asia/Kolkata"));
591603
}
604+
}
592605

606+
public void testStartupParameterPrecedence() throws Exception {
607+
// Test the precedence between guc setting specified through "options" key and one specified
608+
// directly in the startup packet. pgJDBC driver only allows us to use the former method.
609+
// See getParametersForStartup() function in pgJDBC driver code.
610+
//
611+
// To still test for the precedence, we try to set "client_encoding" through "options". The
612+
// driver internally adds a (client_encoding, UTF8) pair to the startup packet and we test that
613+
// that setting has a higher precedence
614+
//
615+
// Note: We don't test that setting an option through "options" field works, that is
616+
// accomplished in testStartupParameters() above
617+
618+
SessionParameter sp =
619+
new SessionParameter("client_encoding", "SQL_ASCII", "UTF8", new ExceptionType[] {});
620+
String parameterName = sp.parameterName;
621+
String expectedValue = sp.expectedValue;
622+
623+
try (Connection conn = getConnectionBuilder()
624+
.withConnectionEndpoint(ConnectionEndpoint.YSQL_CONN_MGR)
625+
.withOptions("-c client_encoding=SQL_ASCII")
626+
.connect();
627+
Statement stmt = conn.createStatement()) {
628+
String fetchedValue = fetchParameterValue(stmt, sp);
629+
assertTrue(String.format("expected value %s for %s, but fetched %s instead", expectedValue,
630+
parameterName, fetchedValue),
631+
expectedValue.equals(fetchedValue));
632+
}
633+
}
634+
635+
@Test
636+
public void testStartupParameterCaseInsensitivity() throws Exception {
637+
// Test that sending a startup parameter in a different case than what Postgres sends in
638+
// ParameterStatus works correctly.
639+
// This mainly tests that whatever state the connection manager stores is case-insensitive:
640+
// either by lowercasing all GUC names in case of auth passthrough or forwarding the startup
641+
// packet to auth backend and using the name in returned ParameterStatus packet.
642+
SessionParameter sp =
643+
new SessionParameter("default_transaction_read_only", "off", "on", new ExceptionType[] {});
644+
String parameterName = sp.parameterName;
645+
String expectedValue = sp.expectedValue;
646+
647+
try (Connection conn = getConnectionBuilder()
648+
.withConnectionEndpoint(ConnectionEndpoint.YSQL_CONN_MGR)
649+
.withOptions("-c DEFAULT_TRANSACTION_READ_ONLY=on")
650+
.connect();
651+
Statement stmt = conn.createStatement()) {
652+
String fetchedValue = fetchParameterValue(stmt, sp);
653+
654+
assertTrue(String.format("expected value %s for %s, but fetched %s instead", expectedValue,
655+
parameterName, fetchedValue),
656+
expectedValue.equals(fetchedValue));
657+
}
658+
}
659+
660+
@Test
661+
public void testInvalidStartupParameter() throws Exception {
662+
// Test that setting an invalid parameter in startup packet fails at the time of database
663+
// connection. We use ysqlsh here since we don't want to execute ANY query after connecting,
664+
// whereas JDBC tries to execute some SET statements after connecting. With JDBC, we try to
665+
// deploy the state to a transactional backend which fails and we're not able to distinguish
666+
// it from connection failure.
667+
668+
// Sleep for some time to wait for the cluster to come up. We have to do this since this
669+
// driver doesn't have any retry mechanism
670+
Thread.sleep(5000);
671+
672+
File pgBinDir = new File(TestUtils.getBuildRootDir(), "postgres/bin");
673+
File ysqlshExec = new File(pgBinDir, "ysqlsh");
674+
final InetSocketAddress postgresAddress = miniCluster.getYsqlConnMgrContactPoints().get(0);
675+
676+
List<String> args = Arrays.asList(
677+
ysqlshExec.toString(),
678+
"-h", postgresAddress.getHostName(),
679+
"-p", Integer.toString(postgresAddress.getPort()),
680+
"-U", "yugabyte",
681+
"-v", "ON_ERROR_STOP=1",
682+
"-c", "\\q"
683+
);
684+
685+
686+
Map<String, String> invalidEnv = new HashMap<String, String>();
687+
invalidEnv.put("PGOPTIONS", "-c geqo=abc");
688+
assertThrows(
689+
IOException.class, () -> { ProcessUtil.runProcess(args, Integer.MAX_VALUE, invalidEnv); });
690+
691+
Map<String, String> validEnv = new HashMap<String, String>();
692+
invalidEnv.put("PGOPTIONS", "-c geqo=off");
693+
ProcessUtil.runProcess(args, Integer.MAX_VALUE, validEnv);
694+
}
695+
696+
@Test
697+
public void testSettingOverrideVariableInStartupPacket() throws Exception {
698+
// Startup parameters are set with the source PGC_S_CLIENT while SET statements are set
699+
// with the source PGC_S_SESSION. There can be settings with PGC_S_OVERRIDE which would
700+
// not be overridden with startup options but would get overriden with SET statements.
701+
// We test this fact with the GUC session_authorization.
702+
703+
final String roleName = "test_user_session_auth";
704+
705+
// Connect as yugabyte user to yugabyte db
593706
try (
594-
Connection conn = getConnectionBuilder()
595-
.withConnectionEndpoint(ConnectionEndpoint.YSQL_CONN_MGR).connect();
707+
Connection conn =
708+
getConnectionBuilder().withConnectionEndpoint(ConnectionEndpoint.YSQL_CONN_MGR)
709+
.withUser("yugabyte").withDatabase("yugabyte").connect();
596710
Statement stmt = conn.createStatement()) {
597-
stmt.execute("SET TimeZone = 'Asia/Kolkata'");
598-
PGConnection pgConn = (PGConnection) conn;
599-
Map<String, String> params = new HashMap<>(pgConn.getParameterStatuses());
711+
stmt.execute(String.format("CREATE ROLE %s WITH login", roleName));
712+
}
600713

601-
// Get the timezone directly
602-
String backendTimeZone = params.get("TimeZone");
603-
assertTrue("Backend timezone should be set correctly from SET command",
604-
backendTimeZone.equals("Asia/Kolkata"));
714+
// Connect as test user to yugabyte db
715+
try (
716+
Connection connTestUser =
717+
getConnectionBuilder().withConnectionEndpoint(ConnectionEndpoint.YSQL_CONN_MGR)
718+
.withUser(roleName).withDatabase("yugabyte").connect();
719+
Statement stmtTestUser = connTestUser.createStatement()) {
720+
try (ResultSet rs = stmtTestUser.executeQuery("SELECT 1")) {
721+
assertTrue("SELECT 1 should return a row", rs.next());
722+
}
605723
}
606724

725+
// Execute a global DDL as a workaround for role not applying when startup packet is processed
726+
try (
727+
Connection conn =
728+
getConnectionBuilder().withConnectionEndpoint(ConnectionEndpoint.YSQL_CONN_MGR)
729+
.withUser("yugabyte").withDatabase("yugabyte").connect();
730+
Statement stmt = conn.createStatement()) {
731+
stmt.execute(
732+
"create role tmp1; grant tmp1 to yugabyte; revoke tmp1 from yugabyte; drop role tmp1;");
733+
}
734+
735+
// Try to set session authorization in startup packet
736+
String sessionAuthOption = String.format("-c session_authorization=%s", roleName);
737+
607738
try (
608739
Connection conn = getConnectionBuilder()
609-
.withConnectionEndpoint(ConnectionEndpoint.YSQL_CONN_MGR).connect();
740+
.withConnectionEndpoint(ConnectionEndpoint.YSQL_CONN_MGR).withUser("yugabyte")
741+
.withDatabase("yugabyte").withOptions(sessionAuthOption).connect();
610742
Statement stmt = conn.createStatement()) {
611-
stmt.execute("SET timezone = 'Asia/Kolkata'");
612-
PGConnection pgConn = (PGConnection) conn;
613-
Map<String, String> params = new HashMap<>(pgConn.getParameterStatuses());
614743

615-
// Get the timezone directly
616-
String backendTimeZone = params.get("TimeZone");
617-
assertTrue("Backend timezone should be set correctly from SET command",
618-
backendTimeZone.equals("Asia/Kolkata"));
744+
try (ResultSet rs = stmt.executeQuery("SHOW SESSION AUTHORIZATION")) {
745+
assertTrue("SHOW SESSION AUTHORIZATION should return a row", rs.next());
746+
String actualSessionAuth = rs.getString(1);
747+
assertEquals("yugabyte", actualSessionAuth);
748+
}
619749
}
620750
}
621751
}

0 commit comments

Comments
 (0)