Skip to content
Commits on Source (11)
  • Carlos Garnacho's avatar
    libtracker-sparql: Avoid URN queries in DBus endpoint notifications · d7c478b7
    Carlos Garnacho authored
    When a connection is proxied through a TrackerEndpointDBus, there are
    possibly 2 levels of TrackerNotifiers, one being used in the service
    itself to trigger the DBus signal, and another on any listening client
    handling those notifications.
    
    However, both of these are doing an additional query to fetch the URN
    for each of the notified elements. Since we only pass ROWIDs in the
    DBus signal to make the signal data as compact as possible, we can
    avoid this query on the service side. Any client listening to
    notifications will perform its own URN queries anyway.
    
    So avoid this piece of pointless busywork in the service side, and
    special case that the internal notifiers used by endpoints should not
    perform the query for URN data.
    
    This does not make anything noticeably faster (perhaps endpoint signal
    emission is a bit more snappy now, if anything) as all querying and
    cursor handling happens lazily in a separate thread fairly detached
    from the rest of the machinery. However it's still less CPU cycles
    used overall.
    d7c478b7
  • Carlos Garnacho's avatar
    libtracker-sparql: Do not attempt to rebuild non-existing FTS tables · e6476933
    Carlos Garnacho authored
    If a database/ontology has no fulltext-indexed properties, we do not
    create a corresponding FTS table. Likewise, parser/locale updates
    should not attempt to update it, or we will get a "SQL logic error"
    poking non-existing tables.
    
    Closes: https://gitlab.gnome.org/GNOME/nautilus/-/issues/2278
    e6476933
  • Carlos Garnacho's avatar
    core: Handle conversion of values for superproperties with differing types · 5bd60c94
    Carlos Garnacho authored
    This is not quite correct, but it is the most straightforward way to sort
    of handle situations in our own Nepomuk ontology that superproperties have
    different rdfs:range than their subproperties.
    
    For these cases, handle the conversion of the value. Since the worst case
    (and the one we happen to exercise) is about converting resource ROWIDs to
    strings, handle this specific situation.
    
    This helps in serialization and deserialization of this data, since the
    ROWIDs were not something that could be replicated in other databases
    without mismatches (e.g. the resource having a different ROWID in the
    new database importing the data) or worse (e.g. cardinality or other
    constraints broken).
    5bd60c94
  • Carlos Garnacho's avatar
    tests: Specify missing order in test · 6c876ead
    Carlos Garnacho authored
    The order of the returned resultset was implicit and up to SQLite, but
    the order for this test started changing starting with SQLite 3.39.0.
    Make the order explicit, so that SQLite implementation details don't
    leak up here.
    
    Closes: https://gitlab.gnome.org/GNOME/tracker/-/issues/370
    6c876ead
  • Sam Thursfield's avatar
    build: CLI depends on libtracker-common · 76177006
    Sam Thursfield authored
    Fix occasional build failures such as this one:
    https://gitlab.gnome.org/GNOME/tracker/-/jobs/2101449
    76177006
  • Carlos Garnacho's avatar
    libtracker-common: Drop unused enum · 06d87e6d
    Carlos Garnacho authored
    And GType boilerplate generation. This enum is entirely unused
    in the tracker code and is thus unneeded.
    06d87e6d
  • Carlos Garnacho's avatar
    libtracker-data: Fetch nrl:modified from all graphs · 65c80a4d
    Carlos Garnacho authored
    We are just looking at the default graph when initializing nrl:modified
    after opening an existing database. Since the default graph possibly
    only contains part of the data, we are possibly/likely missing the latest
    sequence number if used on other graphs.
    
    Check all graphs here relying on our virtual tracker_triples table, so
    we are ensured to get the actual latest/greatest nrl:modified sequence
    that was previously used from the union of all graphs. This ensures us
    to get always a fresh sequence number for newly inserted data.
    65c80a4d
  • Carlos Garnacho's avatar
    53069369
  • Sam Thursfield's avatar
    Merge branch 'wip/carlosg/backports-3.3' into 'tracker-3.3' · 53fba631
    Sam Thursfield authored
    Backports for 3.3
    
    See merge request GNOME/tracker!521
    53fba631
  • Carlos Garnacho's avatar
    Release 3.3.2 · b08f4768
    Carlos Garnacho authored
    b08f4768
  • Jeremy Bicha's avatar
    New upstream version 3.3.2 · 09dfcf6f
    Jeremy Bicha authored
    09dfcf6f
......@@ -387,6 +387,8 @@ test-website:
- website
needs:
- build-fedora-container@x86_64
only:
- master
coverage:
extends:
......
NEW in 3.3.2 - 2022-07-06
=========================
* Avoid redundant queries in TrackerNotifiers proxied through
an D-Bus connection
* Do not attempt to rebuild non-existing FTS tables on parser
updates
* Convert values to the right type when propagating insertions
over superproperties
* Fix test to handle SQLite >= 3.39.0
* Fix handling of nrl:modified after opening existing databases
* Linking fixes to CLI executables
NEW in 3.3.1 - 2022-05-31
=========================
* Fixed blank nodes to return the correct identifier in certain
......
<?xml version='1.0' encoding='UTF-8'?>
<book xmlns="http://www.devhelp.net/book" title="Tracker 3.3.1" link="index.html" author="hotdoc" name="Tracker" version="2" language="C" online="https://gnome.pages.gitlab.gnome.org/tracker/docs/developer/">
<book xmlns="http://www.devhelp.net/book" title="Tracker 3.3.2" link="index.html" author="hotdoc" name="Tracker" version="2" language="C" online="https://gnome.pages.gitlab.gnome.org/tracker/docs/developer/">
<chapters>
<sub name="Overview" link="overview.html"/>
<sub name="Examples" link="examples.html"/>
......
project('tracker', 'c', 'vala',
version: '3.3.1',
version: '3.3.2',
meson_version: '>=0.51',
default_options: [
'c_std=c99',
......
tracker-turtle-writer.c
tracker-turtle-writer.h
tracker-enum-types.[c|h]
tracker-parser-sha1.h
subdir('stop-words')
tracker_common_enums_header = files('tracker-enums.h')
enums = gnome.mkenums('tracker-enum-types',
sources: tracker_common_enums_header,
c_template: 'tracker-enum-types.c.template',
h_template: 'tracker-enum-types.h.template',
)
tracker_common_enum_header = enums[1]
tracker_common_sources = [
'tracker-date-time.c',
'tracker-debug.c',
......@@ -19,7 +9,6 @@ tracker_common_sources = [
'tracker-locale.c',
'tracker-parser-utils.c',
'tracker-language.c',
enums[0], enums[1],
]
if unicode_library_name == 'icu'
......@@ -46,7 +35,6 @@ libtracker_common = static_library('tracker-common',
commoninc = include_directories('.')
tracker_common_dep = declare_dependency(
sources: [tracker_common_enum_header],
link_with: libtracker_common,
dependencies: tracker_common_dependencies,
include_directories: [configinc, srcinc, commoninc],
......
......@@ -36,7 +36,6 @@
#include "tracker-term-utils.h"
#include "tracker-utils.h"
#include "tracker-locale.h"
#include "tracker-enum-types.h"
#undef __LIBTRACKER_COMMON_INSIDE__
......
/*** BEGIN file-header ***/
#include <config.h>
#include "tracker-enum-types.h"
/*** END file-header ***/
/*** BEGIN file-production ***/
/* enumerations from "@basename@" */
#include "@filename@"
/*** END file-production ***/
/*** BEGIN value-header ***/
GType
@enum_name@_get_type (void)
{
static gsize g_define_type_id = 0;
if (g_once_init_enter (&g_define_type_id)) {
static const G@Type@Value values[] = {
/*** END value-header ***/
/*** BEGIN value-production ***/
{ @VALUENAME@, "@VALUENAME@", "@valuenick@" },
/*** END value-production ***/
/*** BEGIN value-tail ***/
{ 0, NULL, NULL }
};
GType type =
g_@type@_register_static (g_intern_static_string ("@EnumName@"), values);
g_once_init_leave (&g_define_type_id, type);
}
return g_define_type_id;
}
/*** END value-tail ***/
/*** BEGIN file-tail ***/
/*** END file-tail ***/
/*** BEGIN file-header ***/
#ifndef __TRACKER_ENUMS_TYPES_H__
#define __TRACKER_ENUMS_TYPES_H__
#include <glib-object.h>
#include "tracker-enums.h"
G_BEGIN_DECLS
/*** END file-header ***/
/*** BEGIN file-production ***/
/* enumerations from "@basename@" */
/*** END file-production ***/
/*** BEGIN value-header ***/
GType @enum_name@_get_type (void) G_GNUC_CONST;
#define TRACKER_TYPE_@ENUMSHORT@ (@enum_name@_get_type ())
/*** END value-header ***/
/*** BEGIN file-tail ***/
G_END_DECLS
#endif /* __TRACKER_ENUMS_TYPES_H__ */
/*** END file-tail ***/
/*
* Copyright (C) 2011, Nokia <ivan.frade@nokia.com>
*
* This library is free software; you can redistribute it and/or
* modify it under the terms of the GNU Lesser General Public
* License as published by the Free Software Foundation; either
* version 2.1 of the License, or (at your option) any later version.
*
* This library is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public
* License along with this library; if not, write to the
* Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor,
* Boston, MA 02110-1301, USA.
*/
#ifndef __TRACKER_ENUMS_H__
#define __TRACKER_ENUMS_H__
G_BEGIN_DECLS
typedef enum {
TRACKER_SERIALIZATION_FORMAT_SPARQL,
TRACKER_SERIALIZATION_FORMAT_TURTLE,
/* JSON and JSON_LD are treated as the same thing right now, but we could
* treat them differently if we wanted. also it's nice to be able to pass
* both 'json' and 'json-ld' to `tracker extract --output-format=`.
*/
TRACKER_SERIALIZATION_FORMAT_JSON,
TRACKER_SERIALIZATION_FORMAT_JSON_LD,
} TrackerSerializationFormat;
G_END_DECLS
#endif /* __TRACKER_ENUMS_H__ */
......@@ -33,7 +33,6 @@ libtracker_data = static_library('tracker-data',
'tracker-uuid.c',
'tracker-vtab-service.c',
'tracker-vtab-triples.c',
tracker_common_enum_header,
tracker_data_enums[0],
tracker_data_enums[1],
c_args: tracker_c_args,
......@@ -50,5 +49,4 @@ tracker_data_dep = declare_dependency(
link_with: [libtracker_data],
dependencies: tracker_data_dependencies,
include_directories: include_directories('.'),
sources: enums[1],
)
......@@ -3831,20 +3831,34 @@ rebuild_fts_tokens (TrackerDataManager *manager,
TrackerDBInterface *iface,
GError **error)
{
TrackerProperty **properties;
GHashTableIter iter;
gchar *graph;
gboolean has_fts = FALSE;
guint len, i;
g_debug ("Rebuilding FTS tokens, this may take a moment...");
if (!tracker_db_interface_sqlite_fts_rebuild_tokens (iface, "main", error))
return FALSE;
properties = tracker_ontologies_get_properties (manager->ontologies, &len);
g_hash_table_iter_init (&iter, manager->graphs);
while (g_hash_table_iter_next (&iter, (gpointer*) &graph, NULL)) {
if (!tracker_db_interface_sqlite_fts_rebuild_tokens (iface, graph, error))
for (i = 0; i < len; i++) {
has_fts |= tracker_property_get_fulltext_indexed (properties[i]);
if (has_fts)
break;
}
if (has_fts) {
g_debug ("Rebuilding FTS tokens, this may take a moment...");
if (!tracker_db_interface_sqlite_fts_rebuild_tokens (iface, "main", error))
return FALSE;
g_hash_table_iter_init (&iter, manager->graphs);
while (g_hash_table_iter_next (&iter, (gpointer*) &graph, NULL)) {
if (!tracker_db_interface_sqlite_fts_rebuild_tokens (iface, graph, error))
return FALSE;
}
g_debug ("FTS tokens rebuilt");
}
g_debug ("FTS tokens rebuilt");
/* Update the stamp file */
tracker_db_manager_tokenizer_update (manager->db_manager);
......
......@@ -406,6 +406,8 @@ tracker_data_update_initialize_modseq (TrackerData *data,
TrackerDBCursor *cursor = NULL;
TrackerDBInterface *temp_iface;
TrackerDBStatement *stmt;
TrackerOntologies *ontologies;
TrackerProperty *property;
GError *inner_error = NULL;
gint max_modseq = 0;
......@@ -414,9 +416,13 @@ tracker_data_update_initialize_modseq (TrackerData *data,
return TRUE;
temp_iface = tracker_data_manager_get_writable_db_interface (data->manager);
ontologies = tracker_data_manager_get_ontologies (data->manager);
property = tracker_ontologies_get_property_by_uri (ontologies, TRACKER_PREFIX_NRL "modified");
stmt = tracker_db_interface_create_statement (temp_iface, TRACKER_DB_STATEMENT_CACHE_TYPE_SELECT, &inner_error,
"SELECT MAX(\"nrl:modified\") AS A FROM \"rdfs:Resource\"");
stmt = tracker_db_interface_create_vstatement (temp_iface, TRACKER_DB_STATEMENT_CACHE_TYPE_SELECT, &inner_error,
"SELECT MAX(object) FROM tracker_triples "
"WHERE predicate = %" G_GINT64_FORMAT,
tracker_property_get_id (property));
if (stmt) {
cursor = tracker_db_statement_start_cursor (stmt, &inner_error);
......@@ -1775,6 +1781,36 @@ process_domain_indexes (TrackerData *data,
}
}
static gboolean
maybe_convert_value (TrackerData *data,
TrackerPropertyType source,
TrackerPropertyType target,
const GValue *value,
GValue *value_out)
{
if (source == TRACKER_PROPERTY_TYPE_RESOURCE &&
target == TRACKER_PROPERTY_TYPE_STRING &&
G_VALUE_HOLDS_INT64 (value)) {
TrackerDBInterface *iface;
gchar *str;
iface = tracker_data_manager_get_writable_db_interface (data->manager);
str = tracker_data_query_resource_urn (data->manager, iface,
g_value_get_int64 (value));
if (!str) {
str = g_strdup_printf ("urn:bnode:%" G_GINT64_FORMAT,
g_value_get_int64 (value));
}
g_value_init (value_out, G_TYPE_STRING);
g_value_take_string (value_out, str);
return TRUE;
}
return FALSE;
}
static gboolean
cache_insert_metadata_decomposed (TrackerData *data,
TrackerProperty *property,
......@@ -1806,6 +1842,8 @@ cache_insert_metadata_decomposed (TrackerData *data,
while (*super_properties) {
gboolean super_is_multi;
GArray *super_old_values;
GValue converted = G_VALUE_INIT;
const GValue *val;
super_is_multi = tracker_property_get_multiple_values (*super_properties);
super_old_values = get_old_property_values (data, *super_properties, &new_error);
......@@ -1817,14 +1855,25 @@ cache_insert_metadata_decomposed (TrackerData *data,
data->resource_buffer->fts_updated |=
tracker_property_get_fulltext_indexed (*super_properties);
if (maybe_convert_value (data,
tracker_property_get_data_type (property),
tracker_property_get_data_type (*super_properties),
object,
&converted))
val = &converted;
else
val = object;
if (super_is_multi || super_old_values->len == 0) {
change |= cache_insert_metadata_decomposed (data, *super_properties, object,
change |= cache_insert_metadata_decomposed (data, *super_properties, val,
&new_error);
if (new_error) {
g_value_unset (&converted);
g_propagate_error (error, new_error);
return FALSE;
}
}
g_value_unset (&converted);
super_properties++;
}
......@@ -1951,8 +2000,21 @@ delete_metadata_decomposed (TrackerData *data,
/* also delete super property values */
super_properties = tracker_property_get_super_properties (property);
while (*super_properties) {
change |= delete_metadata_decomposed (data, *super_properties, object, error);
GValue converted = G_VALUE_INIT;
const GValue *val;
if (maybe_convert_value (data,
tracker_property_get_data_type (property),
tracker_property_get_data_type (*super_properties),
object,
&converted))
val = &converted;
else
val = object;
change |= delete_metadata_decomposed (data, *super_properties, val, error);
super_properties++;
g_value_unset (&converted);
}
return change;
......
......@@ -5,7 +5,6 @@ libtracker_bus = static_library('tracker-bus',
'tracker-bus-fd-cursor.vala',
'tracker-bus-statement.vala',
'../../libtracker-common/libtracker-common.vapi',
tracker_common_enum_header,
c_args: tracker_c_args + [
'-include', 'libtracker-sparql/tracker-private.h',
],
......
......@@ -43,6 +43,7 @@
#include "tracker-endpoint-dbus.h"
#include "tracker-notifier.h"
#include "tracker-notifier-private.h"
#include "tracker-private.h"
#include <gio/gio.h>
......@@ -1008,6 +1009,7 @@ tracker_endpoint_dbus_initable_init (GInitable *initable,
conn = tracker_endpoint_get_sparql_connection (endpoint);
endpoint_dbus->notifier = tracker_sparql_connection_create_notifier (conn);
tracker_notifier_disable_urn_query (endpoint_dbus->notifier);
g_signal_connect (endpoint_dbus->notifier, "events",
G_CALLBACK (notifier_events_cb), endpoint);
......
......@@ -39,4 +39,6 @@ _tracker_notifier_event_cache_push_event (TrackerNotifierEventCache *cache,
void _tracker_notifier_event_cache_flush_events (TrackerNotifierEventCache *cache);
void tracker_notifier_disable_urn_query (TrackerNotifier *notifier);
#endif /* __TRACKER_NOTIFIER_PRIVATE_H__ */
......@@ -74,7 +74,8 @@ struct _TrackerNotifierPrivate {
TrackerSparqlStatement *local_statement;
GAsyncQueue *queue;
gint n_local_statement_slots;
gboolean querying;
guint querying : 1;
guint urn_query_disabled : 1;
GMutex mutex;
};
......@@ -633,7 +634,9 @@ _tracker_notifier_event_cache_flush_events (TrackerNotifierEventCache *cache)
cache->first = g_sequence_get_begin_iter (cache->sequence);
g_async_queue_lock (priv->queue);
if (priv->querying) {
if (priv->urn_query_disabled) {
tracker_notifier_emit_events_in_idle (cache);
} else if (priv->querying) {
g_async_queue_push_unlocked (priv->queue, cache);
} else {
priv->querying = TRUE;
......@@ -946,3 +949,12 @@ tracker_notifier_event_get_urn (TrackerNotifierEvent *event)
g_return_val_if_fail (event != NULL, NULL);
return event->urn;
}
void
tracker_notifier_disable_urn_query (TrackerNotifier *notifier)
{
TrackerNotifierPrivate *priv;
priv = tracker_notifier_get_instance_private (notifier);
priv->urn_query_disabled = TRUE;
}
......@@ -27,7 +27,7 @@ executable(command_name, sources,
],
install: true,
install_rpath: tracker_internal_libs_dir,
dependencies: [tracker_sparql_dep, tracker_data_dep],
dependencies: [tracker_common_dep, tracker_sparql_dep, tracker_data_dep],
include_directories: [commoninc, configinc, srcinc],
)
......
......@@ -7,3 +7,4 @@ SELECT ?v ?w ?v2
OPTIONAL { :x :p ?v2 FILTER(?v = 1) }
}
}
ORDER BY ?v ?w ?v2