In this briefing
- 01Exact full-text retrieval adds a query path beyond vector recall
- 02Self-hosted knowledge-source integration begins to shift from custom pipelines to a managed connection
- 03Document permissions can be diagnosed directly against ingestion results
- 04Index snapshots and recovery enter the same acceptance chain
- →What to watch next
- ↗Sources and verification
- Pinecone full-text search became generally available, combining BM25, text filtering and vector retrieval in the same database; Milvus 3.0.1 also added text fields, re-ranking and collection snapshot recovery capabilities.
- Amazon Bedrock Managed Knowledge Base added a native connector for self-hosted Confluence Data Center, bringing pages, blogs, metadata and incremental synchronisation into managed ingestion.
- AWS added document-level ACL queries and console diagnostics, but these interfaces can only explain the ingested permission state; they cannot replace checks against source-system authorisation or prove that the final answer is correct.
Exact full-text retrieval adds a query path beyond vector recall
At hour 20 on 9 September (Shanghai time), Pinecone announced the general availability of full-text search in its database. The official material says that users can employ BM25 full-text search, text-match filtering and dense or sparse vectors in the same index, together with Lucene query syntax, fuzzy matching, stemming and tokenisation. The release lists support for 18 languages and deployment options including serverless and BYOC.
Milvus 3.0.1 was released earlier and added TEXT field support to the Go client and REST API. It also added L1 re-ranking to search-function chains, and configurable weighted RRF re-ranking to FunctionScore, REST, conventional hybrid search and the Go client. The version includes numerous storage, data-consistency and security fixes, but its release notes provide no comparison with Pinecone using the same data, queries and hardware conditions.
Contract numbers, part numbers, error codes and policy clauses often require exact term matches, while business question answering also needs semantic recall. Hybrid queries provide a common engineering entry point for both needs, but do not establish that retrieval quality improves automatically. Actual quality still depends on the languages, tokenisation rules, filters and no-answer samples in an enterprise corpus, as well as separate retesting of recall, false-positive retrieval and ranking stability.
Self-hosted knowledge-source integration begins to shift from custom pipelines to a managed connection
Within the same observation window, AWS released a native Confluence Data Center data-source connector for Amazon Bedrock Managed Knowledge Base. Customers running self-hosted Confluence Data Center can directly crawl pages and blogs in their spaces. The official description says that the connector handles data crawling, metadata extraction and incremental synchronisation, and allows the scope to be filtered by space or content type.
Integrating Confluence Data Center previously required a custom ingestion pipeline. The new connector reduces this connection and synchronisation work, but the announcement gives no figures for synchronisation latency, deletion-propagation time, attachment-parsing coverage or retrieval performance on complex pages. Nor does it say that the native connector can replace every existing custom transformation.
As the front end of a retrieval chain, a native connector can shorten the basic integration path, but data-source credentials, crawl scope, version retention and deletion synchronisation remain operational responsibilities. Production acceptance therefore covers creation, modification, movement, deletion and permission changes; completing the initial full ingestion is only one state in that process.
Document permissions can be diagnosed directly against ingestion results
AWS subsequently released CheckIngestedDocumentAcl and GetIngestedDocumentAcl for Bedrock Managed Knowledge Base. The former checks whether a particular user can access a particular ingested document, while the latter returns the full ACL attached to the document. The console also gained a Document Access Control area, which can check access by document ID and user email, or retrieve the full ACL by document ID.
AWS positions these capabilities for self-service troubleshooting and auditing. When a user cannot see an expected document, an administrator can determine whether its ingested ACL is misconfigured, reducing cases in which a support ticket must be submitted first. The APIs inspect permissions already attached in the knowledge base; they do not re-confirm authorisation in Confluence or another source system in real time, and they do not prove that a document which passes the permission check will be retrieved correctly or produce a correct answer.
Enterprise RAG permission acceptance can move from inferring behaviour from results to a specific user–document–ACL check, but a complete judgement still depends on the permission versions in the source system and ingestion layer, as well as the propagation time after access is revoked. Diagnostics improve failure explainability, but cannot replace identity governance, least privilege or answer-citation checks.
Index snapshots and recovery enter the same acceptance chain
Milvus 3.0.1 added native collection-level snapshot management and asynchronous recovery through REST v2. It also fixes data-consistency problems in storage paths, reduces repeated scans of authorisation metadata during backup, and addresses cases in which sensitive credentials, API key values, password hashes and external collection-source information could appear in logs or errors. The release notes also increase the bcrypt cost for new password hashes and state explicitly that existing hashes require credential rotation before they can be upgraded.
These capabilities address a different layer from AWS's ACL diagnostics: the former concern index state, recovery and some aspects of storage security, while the latter explain document permissions in a managed knowledge base. The public sources do not show that the two products can be combined directly, nor do they provide a recovery time objective or recovery point objective for snapshot restoration.
Once a retrieval system carries policies, contracts and customer records, whether an index retains the same data version, filtering rules and authorisation metadata after recovery directly affects the auditability of its answers. Snapshot creation, asynchronous recovery, permission consistency and post-rollback queries still need to be tested at the actual data scale; providing a snapshot API does not by itself mean that disaster-recovery requirements have been met.
What to watch next
- Whether Pinecone and Milvus receive independent hybrid-retrieval retesting on the same enterprise corpus, exact identifiers and multilingual conditions.
- Further details on synchronisation latency and failed-retry behaviour for attachments, deletions, page moves and permission changes in the Confluence Data Center connector.
- Whether differences among each source-system ACL record, ingested ACL record, retrieval result, cited document version and post-snapshot-recovery state can be reconciled in one pilot record.
Sources and verification
Golden Data has edited this briefing from the public materials listed above. The original sources govern facts and figures. The enterprise relevance sections are Golden Data editorial analysis and do not constitute an endorsement of any third-party product.
← Back to AI Daily Briefing