Skip to content

Commit e591a96

Browse files
tedyutedyu
committed
[yugabyte#5408] [YSQL] Implement jsonpath .datetime() method
Summary: Original Postgres commit was bffe1bd68457e43925c362d8728ce3b25bdf1c94 Changes from system_views.sql are transferred to yb_system_views.sql Changes from jsonb_jsonpath.out / jsonb_jsonpath.sql are transferred to yb_pg_jsonb_jsonpath.out / yb_pg_jsonb_jsonpath.sql, respectively Changes from jsonpath.out / jsonpath.sql are transferred to yb_pg_jsonpath.out / yb_pg_jsonpath.sql, respectively The oid for additions to pg_proc.dat has been adjusted to 8005 ~ 8009 since the oid values from upstream patch are occupied in YB codebase. The changes involve system catalog and catalog version is not bumped Commit message was: This commit implements jsonpath .datetime() method as it's specified in SQL/JSON standard. There are no-argument and single-argument versions of this method. No-argument version selects first of ISO datetime formats matching input string. Single-argument version accepts template string as its argument. Additionally to .datetime() method itself this commit also implements comparison ability of resulting date and time values. There is some difficulty because exising jsonb_path_*() functions are immutable, while comparison of timezoned and non-timezoned types involves current timezone. At first, current timezone could be changes in session. Moreover, timezones themselves are not immutable and could be updated. This is why we let existing immutable functions throw errors on such non-immutable comparison. In the same time this commit provides jsonb_path_*_tz() functions which are stable and support operations involving timezones. As new functions are added to the system catalog, catversion is bumped. Support of .datetime() method was the only blocker prevents T832 from being marked as supported. sql_features.txt is updated correspondingly. Extracted from original patch by Nikita Glukhov, Teodor Sigaev, Oleg Bartunov. Heavily revised by me. Comments were adjusted by Liudmila Mantrova. Discussion: https://postgr.es/m/fcc6fc6a-b497-f39a-923d-aa34d0c588e8%402ndQuadrant.com Discussion: https://postgr.es/m/CAPpHfdsZgYEra_PeCLGNoXOWYx6iU-S3wF8aX0ObQUcZU%2B4XTw%40mail.gmail.com Author: Alexander Korotkov, Nikita Glukhov, Teodor Sigaev, Oleg Bartunov, Liudmila Mantrova Reviewed-by: Anastasia Lubennikova, Peter Eisentraut Test Plan: Build yugabyte db and run tests via Jenkins Reviewers: mihnea, jason Reviewed By: jason Subscribers: yql Differential Revision: https://phabricator.dev.yugabyte.com/D9550
1 parent d73cbb6 commit e591a96

File tree

20 files changed

+2101
-44
lines changed

20 files changed

+2101
-44
lines changed

‎src/postgres/doc/src/sgml/func.sgml‎

Lines changed: 101 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -11631,16 +11631,6 @@ table2-mapping
1163111631
</para>
1163211632

1163311633
<itemizedlist>
11634-
<listitem>
11635-
<para>
11636-
<literal>.datetime()</literal> item method is not implemented yet
11637-
mainly because immutable <type>jsonpath</type> functions and operators
11638-
cannot reference session timezone, which is used in some datetime
11639-
operations. Datetime support will be added to <type>jsonpath</type>
11640-
in future versions of <productname>PostgreSQL</productname>.
11641-
</para>
11642-
</listitem>
11643-
1164411634
<listitem>
1164511635
<para>
1164611636
A path expression can be a boolean predicate, although the SQL/JSON
@@ -11904,6 +11894,20 @@ table2-mapping
1190411894
<entry><literal>$.z.abs()</literal></entry>
1190511895
<entry><literal>0.3</literal></entry>
1190611896
</row>
11897+
<row>
11898+
<entry><literal>datetime()</literal></entry>
11899+
<entry>Date/time value converted from a string</entry>
11900+
<entry><literal>["2015-8-1", "2015-08-12"]</literal></entry>
11901+
<entry><literal>$[*] ? (@.datetime() &lt; "2015-08-2". datetime())</literal></entry>
11902+
<entry><literal>2015-8-1</literal></entry>
11903+
</row>
11904+
<row>
11905+
<entry><literal>datetime(<replaceable>template</replaceable>)</literal></entry>
11906+
<entry>Date/time value converted from a string using the specified template</entry>
11907+
<entry><literal>["12:30", "18:40"]</literal></entry>
11908+
<entry><literal>$[*].datetime("HH24:MI")</literal></entry>
11909+
<entry><literal>"12:30:00", "18:40:00"</literal></entry>
11910+
</row>
1190711911
<row>
1190811912
<entry><literal>keyvalue()</literal></entry>
1190911913
<entry>
@@ -11921,6 +11925,37 @@ table2-mapping
1192111925
</tgroup>
1192211926
</table>
1192311927

11928+
<note>
11929+
<para>
11930+
The result type of <literal>datetime()</literal> and
11931+
<literal>datetime(<replaceable>template</replaceable>)</literal>
11932+
methods can be <type>date</type>, <type>timetz</type>, <type>time</type>,
11933+
<type>timestamptz</type>, or <type>timestamp</type>.
11934+
Both methods determine the result type dynamically.
11935+
</para>
11936+
<para>
11937+
The <literal>datetime()</literal> method sequentially tries ISO formats
11938+
for <type>date</type>, <type>timetz</type>, <type>time</type>,
11939+
<type>timestamptz</type>, and <type>timestamp</type>. It stops on
11940+
the first matching format and the corresponding data type.
11941+
</para>
11942+
<para>
11943+
The <literal>datetime(<replaceable>template</replaceable>)</literal>
11944+
method determines the result type by the provided template string.
11945+
</para>
11946+
<para>
11947+
The <literal>datetime()</literal> and
11948+
<literal>datetime(<replaceable>template</replaceable>)</literal> methods
11949+
use the same parsing rules as <literal>to_timestamp</literal> SQL
11950+
function does (see <xref linkend="functions-formatting"/>) with three
11951+
exceptions. At first, these methods doesn't allow unmatched template
11952+
patterns. At second, only following separators are allowed in the
11953+
template string: minus sign, period, solidus, comma, apostrophe,
11954+
semicolon, colon and space. At third, separators in the template string
11955+
must exactly match the input string.
11956+
</para>
11957+
</note>
11958+
1192411959
<table id="functions-sqljson-filter-ex-table">
1192511960
<title><type>jsonpath</type> Filter Expression Elements</title>
1192611961
<tgroup cols="5">
@@ -12064,6 +12099,15 @@ table2-mapping
1206412099
</tbody>
1206512100
</tgroup>
1206612101
</table>
12102+
12103+
<note>
12104+
<para>
12105+
When different date/time values are compared, an implicit cast is
12106+
applied. A <type>date</type> value can be cast to <type>timestamp</type>
12107+
or <type>timestamptz</type>, <type>timestamp</type> can be cast to
12108+
<type>timestamptz</type>, and <type>time</type> &mdash; to <type>timetz</type>.
12109+
</para>
12110+
</note>
1206712111
</sect3>
1206812112

1206912113
</sect2>
@@ -12296,7 +12340,7 @@ table2-mapping
1229612340
<para>
1229712341
The <literal>@?</literal> and <literal>@@</literal> operators suppress
1229812342
the following errors: lacking object field or array element, unexpected
12299-
JSON item type, and numeric errors.
12343+
JSON item type, datetime and numeric errors.
1230012344
This behavior might be helpful while searching over JSON document
1230112345
collections of varying structure.
1230212346
</para>
@@ -12565,18 +12609,33 @@ table2-mapping
1256512609
<indexterm>
1256612610
<primary>jsonb_path_exists</primary>
1256712611
</indexterm>
12612+
<indexterm>
12613+
<primary>jsonb_path_exists_tz</primary>
12614+
</indexterm>
1256812615
<indexterm>
1256912616
<primary>jsonb_path_match</primary>
1257012617
</indexterm>
12618+
<indexterm>
12619+
<primary>jsonb_path_match_tz</primary>
12620+
</indexterm>
1257112621
<indexterm>
1257212622
<primary>jsonb_path_query</primary>
1257312623
</indexterm>
12624+
<indexterm>
12625+
<primary>jsonb_path_query_tz</primary>
12626+
</indexterm>
1257412627
<indexterm>
1257512628
<primary>jsonb_path_query_array</primary>
1257612629
</indexterm>
12630+
<indexterm>
12631+
<primary>jsonb_path_query_array_tz</primary>
12632+
</indexterm>
1257712633
<indexterm>
1257812634
<primary>jsonb_path_query_first</primary>
1257912635
</indexterm>
12636+
<indexterm>
12637+
<primary>jsonb_path_query_first_tz</primary>
12638+
</indexterm>
1258012639

1258112640
<table id="functions-json-processing-table">
1258212641
<title>JSON Processing Functions</title>
@@ -12916,6 +12975,9 @@ table2-mapping
1291612975
<para><literal>
1291712976
jsonb_path_exists(target jsonb, path jsonpath [, vars jsonb [, silent bool]])
1291812977
</literal></para>
12978+
<para><literal>
12979+
jsonb_path_exists_tz(target jsonb, path jsonpath [, vars jsonb, silent bool])
12980+
</literal></para>
1291912981
</entry>
1292012982
<entry><type>boolean</type></entry>
1292112983
<entry>
@@ -12936,6 +12998,9 @@ table2-mapping
1293612998
<para><literal>
1293712999
jsonb_path_match(target jsonb, path jsonpath [, vars jsonb, silent bool])
1293813000
</literal></para>
13001+
<para><literal>
13002+
jsonb_path_match_tz(target jsonb, path jsonpath [, vars jsonb, silent bool])
13003+
</literal></para>
1293913004
</entry>
1294013005
<entry><type>boolean</type></entry>
1294113006
<entry>
@@ -12957,6 +13022,9 @@ table2-mapping
1295713022
<para><literal>
1295813023
jsonb_path_query(target jsonb, path jsonpath [, vars jsonb, silent bool])
1295913024
</literal></para>
13025+
<para><literal>
13026+
jsonb_path_query_tz(target jsonb, path jsonpath [, vars jsonb, silent bool])
13027+
</literal></para>
1296013028
</entry>
1296113029
<entry><type>setof jsonb</type></entry>
1296213030
<entry>
@@ -12985,6 +13053,9 @@ table2-mapping
1298513053
<para><literal>
1298613054
jsonb_path_query_array(target jsonb, path jsonpath [, vars jsonb, silent bool])
1298713055
</literal></para>
13056+
<para><literal>
13057+
jsonb_path_query_array_tz(target jsonb, path jsonpath [, vars jsonb, silent bool])
13058+
</literal></para>
1298813059
</entry>
1298913060
<entry><type>jsonb</type></entry>
1299013061
<entry>
@@ -13005,6 +13076,9 @@ table2-mapping
1300513076
<para><literal>
1300613077
jsonb_path_query_first(target jsonb, path jsonpath [, vars jsonb, silent bool])
1300713078
</literal></para>
13079+
<para><literal>
13080+
jsonb_path_query_first_tz(target jsonb, path jsonpath [, vars jsonb, silent bool])
13081+
</literal></para>
1300813082
</entry>
1300913083
<entry><type>jsonb</type></entry>
1301013084
<entry>
@@ -13147,11 +13221,8 @@ table2-mapping
1314713221

1314813222
<note>
1314913223
<para>
13150-
The <literal>jsonb_path_exists</literal>, <literal>jsonb_path_match</literal>,
13151-
<literal>jsonb_path_query</literal>, <literal>jsonb_path_query_array</literal>, and
13152-
<literal>jsonb_path_query_first</literal>
13153-
functions have optional <literal>vars</literal> and <literal>silent</literal>
13154-
arguments.
13224+
The <literal>jsonb_path_*</literal> functions have optional
13225+
<literal>vars</literal> and <literal>silent</literal> arguments.
1315513226
</para>
1315613227
<para>
1315713228
If the <parameter>vars</parameter> argument is specified, it provides an
@@ -13165,6 +13236,20 @@ table2-mapping
1316513236
</para>
1316613237
</note>
1316713238

13239+
<note>
13240+
<para>
13241+
Some of the <literal>jsonb_path_*</literal> functions have the
13242+
<literal>_tz</literal> suffix. These functions have been implemented to
13243+
support comparison of date/time values that involves implicit
13244+
timezone-aware casts. Since operations with time zones are not immutable,
13245+
these functions are qualified as stable. Their counterparts without the
13246+
suffix do not support such casts, so they are immutable and can be used for
13247+
such use-cases as expression indexes
13248+
(see <xref linkend="indexes-expressional"/>). There is no difference
13249+
between these functions for other <type>jsonpath</type> operations.
13250+
</para>
13251+
</note>
13252+
1316813253
<para>
1316913254
See also <xref linkend="functions-aggregate"/> for the aggregate
1317013255
function <function>json_agg</function> which aggregates record

‎src/postgres/src/backend/catalog/sql_features.txt‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -507,6 +507,7 @@ T652 SQL-dynamic statements in SQL routines NO
507507
T653 SQL-schema statements in external routines NO
508508
T654 SQL-dynamic statements in external routines NO
509509
T655 Cyclically dependent routines YES
510+
T832 SQL/JSON path language: item method YES
510511
M001 Datalinks NO
511512
M002 Datalinks via SQL/CLI NO
512513
M003 Datalinks via Embedded SQL NO

‎src/postgres/src/backend/catalog/system_views.sql‎

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1159,6 +1159,46 @@ LANGUAGE INTERNAL
11591159
STRICT IMMUTABLE PARALLEL SAFE
11601160
AS 'jsonb_path_query_first';
11611161

1162+
CREATE OR REPLACE FUNCTION
1163+
jsonb_path_exists_tz(target jsonb, path jsonpath, vars jsonb DEFAULT '{}',
1164+
silent boolean DEFAULT false)
1165+
RETURNS boolean
1166+
LANGUAGE INTERNAL
1167+
STRICT STABLE PARALLEL SAFE
1168+
AS 'jsonb_path_exists_tz';
1169+
1170+
CREATE OR REPLACE FUNCTION
1171+
jsonb_path_match_tz(target jsonb, path jsonpath, vars jsonb DEFAULT '{}',
1172+
silent boolean DEFAULT false)
1173+
RETURNS boolean
1174+
LANGUAGE INTERNAL
1175+
STRICT STABLE PARALLEL SAFE
1176+
AS 'jsonb_path_match_tz';
1177+
1178+
CREATE OR REPLACE FUNCTION
1179+
jsonb_path_query_tz(target jsonb, path jsonpath, vars jsonb DEFAULT '{}',
1180+
silent boolean DEFAULT false)
1181+
RETURNS SETOF jsonb
1182+
LANGUAGE INTERNAL
1183+
STRICT STABLE PARALLEL SAFE
1184+
AS 'jsonb_path_query_tz';
1185+
1186+
CREATE OR REPLACE FUNCTION
1187+
jsonb_path_query_array_tz(target jsonb, path jsonpath, vars jsonb DEFAULT '{}',
1188+
silent boolean DEFAULT false)
1189+
RETURNS jsonb
1190+
LANGUAGE INTERNAL
1191+
STRICT STABLE PARALLEL SAFE
1192+
AS 'jsonb_path_query_array_tz';
1193+
1194+
CREATE OR REPLACE FUNCTION
1195+
jsonb_path_query_first_tz(target jsonb, path jsonpath, vars jsonb DEFAULT '{}',
1196+
silent boolean DEFAULT false)
1197+
RETURNS jsonb
1198+
LANGUAGE INTERNAL
1199+
STRICT STABLE PARALLEL SAFE
1200+
AS 'jsonb_path_query_first_tz';
1201+
11621202
--
11631203
-- The default permissions for functions mean that anyone can execute them.
11641204
-- A number of functions shouldn't be executable by just anyone, but rather

‎src/postgres/src/backend/catalog/yb_system_views.sql‎

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1159,6 +1159,46 @@ LANGUAGE INTERNAL
11591159
STRICT IMMUTABLE PARALLEL SAFE
11601160
AS 'jsonb_path_query_first';
11611161

1162+
CREATE OR REPLACE FUNCTION
1163+
jsonb_path_exists_tz(target jsonb, path jsonpath, vars jsonb DEFAULT '{}',
1164+
silent boolean DEFAULT false)
1165+
RETURNS boolean
1166+
LANGUAGE INTERNAL
1167+
STRICT STABLE PARALLEL SAFE
1168+
AS 'jsonb_path_exists_tz';
1169+
1170+
CREATE OR REPLACE FUNCTION
1171+
jsonb_path_match_tz(target jsonb, path jsonpath, vars jsonb DEFAULT '{}',
1172+
silent boolean DEFAULT false)
1173+
RETURNS boolean
1174+
LANGUAGE INTERNAL
1175+
STRICT STABLE PARALLEL SAFE
1176+
AS 'jsonb_path_match_tz';
1177+
1178+
CREATE OR REPLACE FUNCTION
1179+
jsonb_path_query_tz(target jsonb, path jsonpath, vars jsonb DEFAULT '{}',
1180+
silent boolean DEFAULT false)
1181+
RETURNS SETOF jsonb
1182+
LANGUAGE INTERNAL
1183+
STRICT STABLE PARALLEL SAFE
1184+
AS 'jsonb_path_query_tz';
1185+
1186+
CREATE OR REPLACE FUNCTION
1187+
jsonb_path_query_array_tz(target jsonb, path jsonpath, vars jsonb DEFAULT '{}',
1188+
silent boolean DEFAULT false)
1189+
RETURNS jsonb
1190+
LANGUAGE INTERNAL
1191+
STRICT STABLE PARALLEL SAFE
1192+
AS 'jsonb_path_query_array_tz';
1193+
1194+
CREATE OR REPLACE FUNCTION
1195+
jsonb_path_query_first_tz(target jsonb, path jsonpath, vars jsonb DEFAULT '{}',
1196+
silent boolean DEFAULT false)
1197+
RETURNS jsonb
1198+
LANGUAGE INTERNAL
1199+
STRICT STABLE PARALLEL SAFE
1200+
AS 'jsonb_path_query_first_tz';
1201+
11621202
--
11631203
-- The default permissions for functions mean that anyone can execute them.
11641204
-- A number of functions shouldn't be executable by just anyone, but rather

‎src/postgres/src/backend/utils/adt/jsonpath.c‎

Lines changed: 20 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -336,12 +336,14 @@ flattenJsonPathParseItem(StringInfo buf, JsonPathParseItem *item,
336336
case jpiPlus:
337337
case jpiMinus:
338338
case jpiExists:
339+
case jpiDatetime:
339340
{
340341
int32 arg = reserveSpaceForItemPointer(buf);
341342

342-
chld = flattenJsonPathParseItem(buf, item->value.arg,
343-
nestingLevel + argNestingLevel,
344-
insideArraySubscript);
343+
chld = !item->value.arg ? pos :
344+
flattenJsonPathParseItem(buf, item->value.arg,
345+
nestingLevel + argNestingLevel,
346+
insideArraySubscript);
345347
*(int32 *) (buf->data + arg) = chld - pos;
346348
}
347349
break;
@@ -691,6 +693,15 @@ printJsonPathItem(StringInfo buf, JsonPathItem *v, bool inKey,
691693
case jpiDouble:
692694
appendBinaryStringInfo(buf, ".double()", 9);
693695
break;
696+
case jpiDatetime:
697+
appendBinaryStringInfo(buf, ".datetime(", 10);
698+
if (v->content.arg)
699+
{
700+
jspGetArg(v, &elem);
701+
printJsonPathItem(buf, &elem, false, false);
702+
}
703+
appendStringInfoChar(buf, ')');
704+
break;
694705
case jpiKeyValue:
695706
appendBinaryStringInfo(buf, ".keyvalue()", 11);
696707
break;
@@ -753,6 +764,8 @@ jspOperationName(JsonPathItemType type)
753764
return "floor";
754765
case jpiCeiling:
755766
return "ceiling";
767+
case jpiDatetime:
768+
return "datetime";
756769
default:
757770
elog(ERROR, "unrecognized jsonpath item type: %d", type);
758771
return NULL;
@@ -889,6 +902,7 @@ jspInitByBuffer(JsonPathItem *v, char *base, int32 pos)
889902
case jpiPlus:
890903
case jpiMinus:
891904
case jpiFilter:
905+
case jpiDatetime:
892906
read_int32(v->content.arg, base, pos);
893907
break;
894908
case jpiIndexArray:
@@ -913,7 +927,8 @@ jspGetArg(JsonPathItem *v, JsonPathItem *a)
913927
v->type == jpiIsUnknown ||
914928
v->type == jpiExists ||
915929
v->type == jpiPlus ||
916-
v->type == jpiMinus);
930+
v->type == jpiMinus ||
931+
v->type == jpiDatetime);
917932

918933
jspInitByBuffer(a, v->base, v->content.arg);
919934
}
@@ -961,6 +976,7 @@ jspGetNext(JsonPathItem *v, JsonPathItem *a)
961976
v->type == jpiFloor ||
962977
v->type == jpiCeiling ||
963978
v->type == jpiDouble ||
979+
v->type == jpiDatetime ||
964980
v->type == jpiKeyValue ||
965981
v->type == jpiStartsWith);
966982

0 commit comments

Comments
 (0)