Repository navigation
nodeindex:build only based on default dimension #336
Description
We have a setup with a dimension "zz" that only shares the rootpage for language selection, but is defaultPreset.
In the latest version of the Adaptor (on Neos 5.2 & ES 7) nodeindex:build will index only the nodes that are found in the zz dimension and their respective dimension representatives. So we end up with only Home being indexed in all dimensions, but no further pages.
A change introduced that we found is in NodeIndexCommandController:
Instead of calling $this->workspaceIndexer->index(), $this->workspaceIndexer->indexWithDimensions() is being called, which doesn't loop over dimensions initially.
But in order for the nodes to be indexed independently in all dimensions, it still doesnt work properly, since the rootNode still is "dimensionless":
Activity
daniellienert commentedon Jun 18, 2020
Uh that sounds bad. The problem was, that Dimensions were cycled twice, once in the controller and once in https://github.com/Flowpack/Flowpack.ElasticSearch.ContentRepositoryAdaptor/blob/master/Classes/Indexer/NodeIndexer.php#L308
So I checked both loops and removed one of them. But as I understand your setup, traversing the tree is also not possible if the controller would loop over dimensions.
So how did the old version found your nodes?
regniets commentedon Jun 18, 2020
Yes, i still haven't found out why and how it worked in the old version.
The new approach seems to work only if you have 1:1 relations in dimensions and nodes, but doesn't take "loose dimensions" into account. I can provide you with a PR how we worked around this issue (which is very pragmatic, not sure if it's the right way to go) and might make the second loop redundant (you might lose some weird orphan nodes though).
(i.e. dimensions taken into account on root level, not on node level)
daniellienert commentedon Jun 19, 2020
After thinking about. Really helpful would be to add a lightweight test scenario with some nodes, that resamples your setup. The test could just run the indexer and test the result. An example can be found here https://github.com/Flowpack/Flowpack.ElasticSearch.ContentRepositoryAdaptor/blob/master/Tests/Functional/Eel/ElasticSearchQueryTest.php#L407. That would make debugging and fixing it for good much easier.
regniets commentedon Jun 19, 2020
Sounds good. Is there a practical guide on running the tests?
regniets commentedon Jun 19, 2020
Kinda like this?
https://github.com/regniets/Flowpack.ElasticSearch.ContentRepositoryAdaptor/blob/bugfix/multi-dimension-test/Tests/Functional/Indexer/NodeIndexerDimensionsTest.php
Not refined and not tested, that would be my next task :)
daniellienert commentedon Jun 20, 2020
Yeah!
Just open a PR and the test will run automatically. (Will fail because of createNodesForDimensionsTest != createNodesForLanguageDimensions).
You can run the test locally by running
regniets commentedon Jun 22, 2020
Ok, the test fails (as it should, but not because of the wrong method). Took out the var_dump, just wanted to check if the error is the same as i keep getting and it is, so my setup seems to be somewhat correct.
Actually not the error we keep getting in our project, so something might be still wrong in how i adress the problem - although the error is somewhat related and it does state
Workspace "live" without dimensions done, which is definitely wrong.It should have indexed the dimension
['language' => 'zz']for the hash20cc3bfb2e34875a938ea2b8015f24ebregniets commentedon Jun 22, 2020
This now is a (probably) working test scenario:
#338
I had to adjust it to the neos/demo package which is the basis of the functional tests and its dimension-presets.
regniets commentedon Jun 23, 2020
btw - it is not only
nodeindex:build, but alsonodeindex:indexnode, i'll try to cover this one as well...DrillSergeant commentedon Aug 5, 2020
I am also experiencing this behaviour. Our setup is:
Behaviour: Only nodes are indexed that are in the default dimension combination "en" and the variants of them in other dimension.
But nodes that only exists in other languages - and not in the default language - are not indexed.
By changing
$this->workspaceIndexer->indexWithDimensions($workspace, $dimensionsValuesArray, $limit, $workspaceLogger);to$this->workspaceIndexer->index($workspace, $limit, $workspaceLogger);the problem seems to be solved.As well not sure if this (re-)introduces redundant indexing of nodes, which is a big issue in the (old) fork-version of this package.
DrillSergeant commentedon Sep 8, 2020
With the merged PR #350 (version 7.0.3) this issue is gone in my current project.