Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement function timezone() #1386

Merged
merged 46 commits into from
Jul 15, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
46 commits
Select commit Hold shift + click to select a range
99da434
Added conversion str to int
realHannes Apr 25, 2024
424d023
templated function for toNumeric, add declaration to NaryExpression.h
realHannes Apr 25, 2024
0117e82
str to num for SparqlExpression implemented + added test
realHannes Apr 25, 2024
35fd0b1
Merge branch 'ad-freiburg:master' into master
realHannes Apr 26, 2024
94356c2
Update src/engine/sparqlExpressions/StringExpressions.cpp
realHannes Apr 26, 2024
decc8ba
Update src/engine/sparqlExpressions/StringExpressions.cpp
realHannes Apr 26, 2024
850152c
Update src/engine/sparqlExpressions/StringExpressions.cpp
realHannes Apr 26, 2024
d650d67
Update src/engine/sparqlExpressions/StringExpressions.cpp
realHannes Apr 26, 2024
46cc697
using now absl::from_chars() and stripping whitespaces for string to …
realHannes Apr 26, 2024
7fc5c28
added new functions to processIriFuntionCall() (for string to number)
realHannes Apr 26, 2024
efb0e24
renaming to: toIntExpression and toDoubleExpression for later more ge…
realHannes Apr 26, 2024
a88537c
made format (clang-format-16)
realHannes Apr 26, 2024
ca1e2e0
Update src/parser/sparqlParser/SparqlQleverVisitor.cpp
realHannes Apr 29, 2024
4adc831
Update src/parser/sparqlParser/SparqlQleverVisitor.cpp
realHannes Apr 29, 2024
d0f0d63
renaming in NaryExpression.h for accordance with other function, addi…
realHannes Apr 29, 2024
a118609
added test coverage for function calls makeIntExpression and make Dou…
realHannes Apr 29, 2024
062052e
toNumeric has now correct behavior and uses absl::from_chars() and st…
realHannes Apr 29, 2024
6d0f42a
made clang-format for NaryExpressionImpl.h
realHannes Apr 29, 2024
f90b8e2
Merge branch 'ad-freiburg:master' into master
realHannes May 6, 2024
fb88493
Merge branch 'ad-freiburg:master' into master
realHannes May 15, 2024
b2eb514
Merge remote-tracking branch 'upstream/master'
realHannes May 16, 2024
b165ac1
Merge branch 'ad-freiburg:master' into master
realHannes Jun 1, 2024
7a3dfb2
Merge branch 'master' of https://github.com/realHannes/qlever
realHannes Jun 1, 2024
fc0ad3a
Merge branch 'ad-freiburg:master' into master
realHannes Jun 6, 2024
f3e6086
Merge branch 'ad-freiburg:master' into master
realHannes Jun 7, 2024
fd4c351
Merge branch 'ad-freiburg:master' into master
realHannes Jun 10, 2024
220c9bf
Merge branch 'ad-freiburg:master' into master
realHannes Jun 12, 2024
a81cb8a
Merge branch 'ad-freiburg:master' into master
realHannes Jun 13, 2024
acc0109
Merge branch 'ad-freiburg:master' into master
realHannes Jun 14, 2024
cb8e560
Merge branch 'ad-freiburg:master' into master
realHannes Jun 20, 2024
2b39970
Merge branch 'ad-freiburg:master' into master
realHannes Jun 28, 2024
a0101e4
Merge branch 'ad-freiburg:master' into master
realHannes Jun 30, 2024
2d3861b
implemented expression timezone() + anility to compare w.r.t. timezone
realHannes Jun 30, 2024
8432c20
added class for DayTimeDuration (xsd:dayTimeDuration)
realHannes Jul 8, 2024
ed7fad5
Merge branch 'master' into add_timezone_expression
realHannes Jul 8, 2024
f951979
fix merge conflict
realHannes Jul 8, 2024
f876cde
remove std::move
realHannes Jul 8, 2024
a5e5909
one more build fix
realHannes Jul 8, 2024
b0b054e
improve / fix tests
realHannes Jul 8, 2024
1b5ec54
small change to test
realHannes Jul 9, 2024
b52a517
use absl::StrFormat + changes for sonarcloud
realHannes Jul 9, 2024
9302a8d
throw DurationOverFlowException in correct namespace
realHannes Jul 9, 2024
bf3a24f
applied proposed changes
realHannes Jul 12, 2024
caa9b4d
use initialization list
realHannes Jul 12, 2024
0fdd658
update src/util/Duration.h
realHannes Jul 15, 2024
52fecaa
update src/util/Duration.h
realHannes Jul 15, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
77 changes: 54 additions & 23 deletions src/engine/sparqlExpressions/DateExpressions.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -6,17 +6,24 @@

namespace sparqlExpression {
namespace detail {

using LiteralOrIri = ad_utility::triple_component::LiteralOrIri;
using Literal = ad_utility::triple_component::Literal;

// Date functions.
// The input is `std::nullopt` if the argument to the expression is not a date.
inline auto extractYear = [](std::optional<DateOrLargeYear> d) {

//______________________________________________________________________________
inline auto extractYear = [](std::optional<DateYearOrDuration> d) {
if (!d.has_value()) {
return Id::makeUndefined();
} else {
return Id::makeFromInt(d->getYear());
}
};

inline auto extractMonth = [](std::optional<DateOrLargeYear> d) {
//______________________________________________________________________________
inline auto extractMonth = [](std::optional<DateYearOrDuration> d) {
// TODO<C++23> Use the monadic operations for std::optional
if (!d.has_value()) {
return Id::makeUndefined();
Expand All @@ -28,7 +35,8 @@ inline auto extractMonth = [](std::optional<DateOrLargeYear> d) {
return Id::makeFromInt(optionalMonth.value());
};

inline auto extractDay = [](std::optional<DateOrLargeYear> d) {
//______________________________________________________________________________
inline auto extractDay = [](std::optional<DateYearOrDuration> d) {
// TODO<C++23> Use the monadic operations for `std::optional`.
if (!d.has_value()) {
return Id::makeUndefined();
Expand All @@ -40,44 +48,63 @@ inline auto extractDay = [](std::optional<DateOrLargeYear> d) {
return Id::makeFromInt(optionalDay.value());
};

using LiteralOrIri = ad_utility::triple_component::LiteralOrIri;
using Literal = ad_utility::triple_component::Literal;

//______________________________________________________________________________
inline auto extractStrTimezone =
[](std::optional<DateOrLargeYear> d) -> IdOrLiteralOrIri {
if (d.has_value()) {
auto timezoneStr = d.value().getStrTimezone();
return LiteralOrIri{Literal::literalWithNormalizedContent(
asNormalizedStringViewUnsafe(timezoneStr))};
}
return Id::makeUndefined();
};

template <auto dateMember, auto makeId>
inline auto extractTimeComponentImpl = [](std::optional<DateOrLargeYear> d) {
if (!d.has_value() || !d->isDate()) {
return Id::makeUndefined();
}
Date date = d.value().getDate();
if (!date.hasTime()) {
[](std::optional<DateYearOrDuration> d) -> IdOrLiteralOrIri {
// TODO<C++23> Use the monadic operations for std::optional
if (!d.has_value()) {
return Id::makeUndefined();
}
return std::invoke(makeId, std::invoke(dateMember, date));
auto timezoneStr = d.value().getStrTimezone();
return LiteralOrIri{Literal::literalWithNormalizedContent(
asNormalizedStringViewUnsafe(timezoneStr))};
};

//______________________________________________________________________________
inline auto extractTimezoneDurationFormat =
realHannes marked this conversation as resolved.
Show resolved Hide resolved
[](std::optional<DateYearOrDuration> d) {
// TODO<C++23> Use the monadic operations for std::optional
if (!d.has_value()) {
return Id::makeUndefined();
}
const auto& optDayTimeDuration =
DateYearOrDuration::xsdDayTimeDurationFromDate(d.value());
return optDayTimeDuration.has_value()
? Id::makeFromDate(optDayTimeDuration.value())
: Id::makeUndefined();
};

//______________________________________________________________________________
template <auto dateMember, auto makeId>
inline const auto extractTimeComponentImpl =
[](std::optional<DateYearOrDuration> d) {
if (!d.has_value() || !d->isDate()) {
return Id::makeUndefined();
}
Date date = d.value().getDate();
if (!date.hasTime()) {
return Id::makeUndefined();
}
return std::invoke(makeId, std::invoke(dateMember, date));
};

//______________________________________________________________________________
constexpr auto extractHours =
extractTimeComponentImpl<&Date::getHour, &Id::makeFromInt>;
constexpr auto extractMinutes =
extractTimeComponentImpl<&Date::getMinute, &Id::makeFromInt>;
constexpr auto extractSeconds =
extractTimeComponentImpl<&Date::getSecond, &Id::makeFromDouble>;

//______________________________________________________________________________
NARY_EXPRESSION(YearExpression, 1, FV<decltype(extractYear), DateValueGetter>);
NARY_EXPRESSION(MonthExpression, 1,
FV<decltype(extractMonth), DateValueGetter>);
NARY_EXPRESSION(DayExpression, 1, FV<decltype(extractDay), DateValueGetter>);
NARY_EXPRESSION(TimezoneStrExpression, 1,
FV<decltype(extractStrTimezone), DateValueGetter>);
NARY_EXPRESSION(TimezoneDurationExpression, 1,
FV<decltype(extractTimezoneDurationFormat), DateValueGetter>);
NARY_EXPRESSION(HoursExpression, 1,
FV<decltype(extractHours), DateValueGetter>);
NARY_EXPRESSION(MinutesExpression, 1,
Expand All @@ -99,6 +126,10 @@ SparqlExpression::Ptr makeTimezoneStrExpression(SparqlExpression::Ptr child) {
return std::make_unique<TimezoneStrExpression>(std::move(child));
}

SparqlExpression::Ptr makeTimezoneExpression(SparqlExpression::Ptr child) {
return std::make_unique<TimezoneDurationExpression>(std::move(child));
}

SparqlExpression::Ptr makeMonthExpression(SparqlExpression::Ptr child) {
return std::make_unique<MonthExpression>(std::move(child));
}
Expand Down
1 change: 1 addition & 0 deletions src/engine/sparqlExpressions/NaryExpression.h
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,7 @@ SparqlExpression::Ptr makeMinutesExpression(SparqlExpression::Ptr child);
SparqlExpression::Ptr makeHoursExpression(SparqlExpression::Ptr child);
SparqlExpression::Ptr makeDayExpression(SparqlExpression::Ptr child);
SparqlExpression::Ptr makeTimezoneStrExpression(SparqlExpression::Ptr child);
SparqlExpression::Ptr makeTimezoneExpression(SparqlExpression::Ptr child);
SparqlExpression::Ptr makeMonthExpression(SparqlExpression::Ptr child);
SparqlExpression::Ptr makeYearExpression(SparqlExpression::Ptr child);

Expand Down
6 changes: 3 additions & 3 deletions src/engine/sparqlExpressions/NowDatetimeExpression.h
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

#include "engine/sparqlExpressions/SparqlExpression.h"
#include "util/ChunkedForLoop.h"
#include "util/Date.h"
#include "util/DateYearDuration.h"
#include "util/Random.h"

namespace sparqlExpression {
Expand All @@ -15,11 +15,11 @@ namespace sparqlExpression {
// query the same.
class NowDatetimeExpression : public SparqlExpression {
private:
DateOrLargeYear date_;
DateYearOrDuration date_;

public:
explicit NowDatetimeExpression(const std::string& dateTimeFormat)
: date_(DateOrLargeYear::parseXsdDatetime(dateTimeFormat)) {}
: date_(DateYearOrDuration::parseXsdDatetime(dateTimeFormat)) {}

std::string getCacheKey(
[[maybe_unused]] const VariableToColumnMap& varColMap) const override {
Expand Down
4 changes: 2 additions & 2 deletions src/engine/sparqlExpressions/SparqlExpressionValueGetters.h
Original file line number Diff line number Diff line change
Expand Up @@ -190,10 +190,10 @@ struct IsNumericValueGetter : Mixin<IsNumericValueGetter> {
};

/// This class can be used as the `ValueGetter` argument of Expression
/// templates. It produces a `std::optional<DateOrLargeYear>`.
/// templates. It produces a `std::optional<DateYearOrDuration>`.
struct DateValueGetter : Mixin<DateValueGetter> {
using Mixin<DateValueGetter>::operator();
using Opt = std::optional<DateOrLargeYear>;
using Opt = std::optional<DateYearOrDuration>;

Opt operator()(ValueId id, const EvaluationContext*) const {
if (id.getDatatype() == Datatype::Date) {
Expand Down
2 changes: 2 additions & 0 deletions src/global/Constants.h
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,8 @@ static const char XSD_DATE_TYPE[] = "http://www.w3.org/2001/XMLSchema#date";
static const char XSD_GYEAR_TYPE[] = "http://www.w3.org/2001/XMLSchema#gYear";
static const char XSD_GYEARMONTH_TYPE[] =
"http://www.w3.org/2001/XMLSchema#gYearMonth";
static const char XSD_DAYTIME_DURATION_TYPE[] =
"http://www.w3.org/2001/XMLSchema#dayTimeDuration";

constexpr inline char XSD_INT_TYPE[] = "http://www.w3.org/2001/XMLSchema#int";
static const char XSD_INTEGER_TYPE[] =
Expand Down
10 changes: 5 additions & 5 deletions src/global/ValueId.h
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@

#include "global/IndexTypes.h"
#include "util/BitUtils.h"
#include "util/Date.h"
#include "util/DateYearDuration.h"
#include "util/NBitInteger.h"
#include "util/Serializer/Serializer.h"
#include "util/SourceLocation.h"
Expand Down Expand Up @@ -257,12 +257,12 @@ class ValueId {
}

// Store or load a `Date` object.
static ValueId makeFromDate(DateOrLargeYear d) noexcept {
static ValueId makeFromDate(DateYearOrDuration d) noexcept {
return addDatatypeBits(std::bit_cast<uint64_t>(d), Datatype::Date);
}

DateOrLargeYear getDate() const noexcept {
return std::bit_cast<DateOrLargeYear>(removeDatatypeBits(_bits));
DateYearOrDuration getDate() const noexcept {
return std::bit_cast<DateYearOrDuration>(removeDatatypeBits(_bits));
}

// TODO<joka921> implement dates
Expand Down Expand Up @@ -340,7 +340,7 @@ class ValueId {
ostr << std::to_string(value);
} else if constexpr (ad_utility::isSimilar<T, bool>) {
ostr << (value ? "true" : "false");
} else if constexpr (ad_utility::isSimilar<T, DateOrLargeYear>) {
} else if constexpr (ad_utility::isSimilar<T, DateYearOrDuration>) {
ostr << value.toStringAndType().first;
} else if constexpr (ad_utility::isSimilar<T, LocalVocabIndex>) {
AD_CORRECTNESS_CHECK(value != nullptr);
Expand Down
8 changes: 4 additions & 4 deletions src/index/IndexFormatVersion.h
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@

#include <cstdint>

#include "util/Date.h"
#include "util/DateYearDuration.h"
#include "util/json.h"

namespace qlever {
Expand All @@ -17,7 +17,7 @@ struct IndexFormatVersion {
// The number of the pull request that changed the index format most recently.
uint64_t prNumber_;
// The date of the last breaking change of the index format.
DateOrLargeYear date_{Date{1900, 1, 1}};
DateYearOrDuration date_{Date{1900, 1, 1}};

// Conversion To JSON.
friend void to_json(nlohmann::json& j, const IndexFormatVersion& version) {
Expand All @@ -28,13 +28,13 @@ struct IndexFormatVersion {
// Conversion from JSON.
friend void from_json(const nlohmann::json& j, IndexFormatVersion& version) {
version.prNumber_ = static_cast<uint64_t>(j["pull-request-number"]);
version.date_ = DateOrLargeYear::parseXsdDate(std::string{j["date"]});
version.date_ = DateYearOrDuration::parseXsdDate(std::string{j["date"]});
}
bool operator==(const IndexFormatVersion&) const = default;
};

// The actual index version. Change it once the binary format of the index
// changes.
inline const IndexFormatVersion& indexFormatVersion{
1320, DateOrLargeYear{Date{2024, 4, 17}}};
1320, DateYearOrDuration{Date{2024, 4, 17}}};
} // namespace qlever
4 changes: 2 additions & 2 deletions src/parser/TripleComponent.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ std::ostream& operator<<(std::ostream& stream, const TripleComponent& obj) {
stream << value.toStringRepresentation();
} else if constexpr (std::is_same_v<T, TripleComponent::Iri>) {
stream << value.toStringRepresentation();
} else if constexpr (std::is_same_v<T, DateOrLargeYear>) {
} else if constexpr (std::is_same_v<T, DateYearOrDuration>) {
stream << "DATE: " << value.toStringAndType().first;
} else if constexpr (std::is_same_v<T, bool>) {
stream << (value ? "true" : "false");
Expand Down Expand Up @@ -53,7 +53,7 @@ std::optional<Id> TripleComponent::toValueIdIfNotString() const {
return Id::makeFromBool(value);
} else if constexpr (std::is_same_v<T, UNDEF>) {
return Id::makeUndefined();
} else if constexpr (std::is_same_v<T, DateOrLargeYear>) {
} else if constexpr (std::is_same_v<T, DateYearOrDuration>) {
return Id::makeFromDate(value);
} else if constexpr (std::is_same_v<T, Variable>) {
// Cannot turn a variable into a ValueId.
Expand Down
2 changes: 1 addition & 1 deletion src/parser/TripleComponent.h
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ class TripleComponent {
private:
// The underlying variant type.
using Variant = std::variant<std::string, double, int64_t, bool, UNDEF,
Variable, Literal, Iri, DateOrLargeYear>;
Variable, Literal, Iri, DateYearOrDuration>;
Variant _variant;

public:
Expand Down
28 changes: 24 additions & 4 deletions src/parser/TurtleParser.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@
#include "parser/NormalizedString.h"
#include "parser/RdfEscaping.h"
#include "util/Conversions.h"
#include "util/DateYearDuration.h"
#include "util/OnDestructionDontThrowDuringStackUnwinding.h"

using namespace std::chrono_literals;
Expand Down Expand Up @@ -417,16 +418,19 @@ TripleComponent TurtleParser<T>::literalAndDatatypeToTripleComponentImpl(
parser->parseDoubleConstant(normalizedLiteralContent);
} else if (type == XSD_DATETIME_TYPE) {
parser->lastParseResult_ =
DateOrLargeYear::parseXsdDatetime(normalizedLiteralContent);
DateYearOrDuration::parseXsdDatetime(normalizedLiteralContent);
} else if (type == XSD_DATE_TYPE) {
parser->lastParseResult_ =
DateOrLargeYear::parseXsdDate(normalizedLiteralContent);
DateYearOrDuration::parseXsdDate(normalizedLiteralContent);
} else if (type == XSD_GYEARMONTH_TYPE) {
parser->lastParseResult_ =
DateOrLargeYear::parseGYearMonth(normalizedLiteralContent);
DateYearOrDuration::parseGYearMonth(normalizedLiteralContent);
} else if (type == XSD_GYEAR_TYPE) {
parser->lastParseResult_ =
DateOrLargeYear::parseGYear(normalizedLiteralContent);
DateYearOrDuration::parseGYear(normalizedLiteralContent);
} else if (type == XSD_DAYTIME_DURATION_TYPE) {
parser->lastParseResult_ =
DateYearOrDuration::parseXsdDayTimeDuration(normalizedLiteralContent);
} else {
literal.addDatatype(typeIri);
parser->lastParseResult_ = std::move(literal);
Expand All @@ -447,6 +451,22 @@ TripleComponent TurtleParser<T>::literalAndDatatypeToTripleComponentImpl(
"instead"
<< std::endl;
parser->lastParseResult_ = std::move(literal);
} catch (const DurationParseException&) {
LOG(DEBUG) << normalizedLiteralContent
<< " could not be parsed as a duration object of type " << type
<< ". It is treated as a plain string literal without datatype "
"instead"
<< std::endl;
parser->lastParseResult_ = std::move(literal);
} catch (const DurationOverflowException& ex) {
LOG(DEBUG) << normalizedLiteralContent
<< " could not be parsed as duration object for the following "
"reason: "
<< ex.what()
<< ". It is treated as a plain string literal without datatype "
"instead"
<< std::endl;
parser->lastParseResult_ = std::move(literal);
} catch (const std::exception& e) {
parser->raise(e.what());
}
Expand Down
1 change: 1 addition & 0 deletions src/parser/TurtleParser.h
Original file line number Diff line number Diff line change
Expand Up @@ -501,6 +501,7 @@ class TurtleStringParser : public TurtleParser<Tokenizer_T> {
FRIEND_TEST(TurtleParserTest, blankNode);
FRIEND_TEST(TurtleParserTest, blankNodePropertyList);
FRIEND_TEST(TurtleParserTest, DateLiterals);
FRIEND_TEST(TurtleParserTest, DayTimeDurationLiterals);
};

/**
Expand Down
2 changes: 2 additions & 0 deletions src/parser/sparqlParser/SparqlQleverVisitor.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -2022,6 +2022,8 @@ ExpressionPtr Visitor::visit([[maybe_unused]] Parser::BuiltInCallContext* ctx) {
return createUnary(&makeDayExpression);
} else if (functionName == "tz") {
return createUnary(&makeTimezoneStrExpression);
} else if (functionName == "timezone") {
return createUnary(&makeTimezoneExpression);
} else if (functionName == "now") {
AD_CONTRACT_CHECK(argList.empty());
return std::make_unique<NowDatetimeExpression>(startTime_);
Expand Down
2 changes: 1 addition & 1 deletion src/util/CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
add_subdirectory(ConfigManager)
add_subdirectory(MemorySize)
add_subdirectory(http)
add_library(util GeoSparqlHelpers.cpp antlr/ANTLRErrorHandling.cpp ParseException.cpp Conversions.cpp Date.cpp antlr/GenerateAntlrExceptionMetadata.cpp CancellationHandle.cpp StringUtils.cpp)
add_library(util GeoSparqlHelpers.cpp antlr/ANTLRErrorHandling.cpp ParseException.cpp Conversions.cpp Date.cpp DateYearDuration.cpp Duration.cpp antlr/GenerateAntlrExceptionMetadata.cpp CancellationHandle.cpp StringUtils.cpp)
qlever_target_link_libraries(util re2::re2)
Loading
Loading