I am unable to find any documentation mentioning how are cosmos db indexes organized per the number of physical partitions. If i have my logical partition split into multiple physical partitions and assuming i am not including a partition key in the filter and have created an index on the field i am querying with. What would the behavior be in terms of index. Does cosmos create an individual index per physical partition or a centrally maintained global index?
Can someone please explain what the behavior could be in such a case or point to some documentation in azure which explains how this would work.
CodePudding user response:
Kind regards
CodePudding user response:
A physical partition is simply a compute and storage node on which your data resides. A partition key within your WHERE clause routes the query to the partition where that data resides. Indexes reside within each partition and index the data for that partition only. Partitions are share nothing. In addition to routing, partition keys must also be included in your index policy when used in queries.
A query without a partition key in the filter will fan out to every partition within a container. At small scales (< 10K RU/s or < 50GB) this isn't much of an issue because the data is all located on a single physical partition. However, as the amount of storage and throughput grows, this query will likely become increasingly more expensive with greater latency. In short, the query will not scale. This is because as the size grows, so does the number of physical partitions that must be contacted to serve the same query.
More information here, Tuning query performance with Azure Cosmos DB and here, Indexing Overview