Skip to content

Commit c01528e

Browse files
rohanshah18rohanshah18
committed
Add createNamespace(), and prefix + totalCount to listNamespaces() (#205)
## Problem Add support for `createNamespace()` endpoint, and add `prefix` + `totalCount` to `listNamespaces()`. ## Solution This PR adds the `createNamespace` method and enhances `listNamespaces` with prefix filtering and total count support for both `Index` and `AsyncIndex` clients. **Create Namespace**: 1. Basic Namespace Creation: Create a namespace explicitly by name: ```java // Create a namespace explicitly NamespaceDescription createdNamespace = index.createNamespace("my-namespace"); assertNotNull(createdNamespace); assertEquals("my-namespace", createdNamespace.getName()); // Verify the namespace was created ListNamespacesResponse response = index.listNamespaces(); assertTrue(response.getNamespacesList().stream() .anyMatch(ns -> ns.getName().equals("my-namespace"))); ``` 2. Metadata Schema Support: Create a namespace with a metadata schema to define filterable fields: ```java // Create a namespace with metadata schema MetadataSchema schema = MetadataSchema.newBuilder() .putFields("genre", MetadataFieldProperties.newBuilder().setFilterable(true).build()) .putFields("year", MetadataFieldProperties.newBuilder().setFilterable(true).build()) .build(); NamespaceDescription namespaceWithSchema = index.createNamespace("movies-namespace", schema); assertNotNull(namespaceWithSchema); assertEquals("movies-namespace", namespaceWithSchema.getName()); ``` **Prefix Filtering**: Filter namespaces by prefix to retrieve only namespaces starting with a specific prefix: ```java // List namespaces with prefix filtering ListNamespacesResponse response = index.listNamespaces("test-", null, 100); int totalCount = response.getTotalCount(); // Total namespaces matching "test-" prefix List<NamespaceDescription> namespaces = response.getNamespacesList(); // Verify all returned namespaces match the prefix for (NamespaceDescription ns : namespaces) { assert ns.getName().startsWith("test-"); } ``` **Total Count**: The response includes a `totalCount` field indicating the total number of namespaces matching the prefix (useful for pagination): ``` // Get total count even when results are paginated ListNamespacesResponse response = index.listNamespaces("prod-", null, 10); int totalCount = response.getTotalCount(); // Total matching namespaces int returnedCount = response.getNamespacesCount(); // Namespaces in this page (≤ 10) // Use totalCount to determine if more pages are available if (totalCount > returnedCount) { // More namespaces available via pagination } ``` **Async Support**: Same functionality available for `AsyncIndex`: ``` // Create namespace asynchronously ListenableFuture<NamespaceDescription> future = asyncIndex.createNamespace("async-namespace"); NamespaceDescription createdNamespace = future.get(); // Create namespace with schema asynchronously ListenableFuture<NamespaceDescription> futureWithSchema = asyncIndex.createNamespace("async-movies", schema); NamespaceDescription namespace = futureWithSchema.get(); // List namespaces with prefix filtering asynchronously ListenableFuture<ListNamespacesResponse> listFuture = asyncIndex.listNamespaces("dev-", null, 50); ListNamespacesResponse response = listFuture.get(); int totalCount = response.getTotalCount(); ``` **Note**: - Null or empty prefix values are ignored (no filtering applied) - The method signature is: `listNamespaces(String prefix, String paginationToken, int limit)` - Both synchronous (`Index`) and asynchronous (`AsyncIndex`) implementations are included - Fully backward compatible - existing `listNamespaces()` methods remain unchanged ## Type of Change - [ ] Bug fix (non-breaking change which fixes an issue) - [X] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) - [ ] This change requires a documentation update - [ ] Infrastructure change (CI configs, etc) - [ ] Non-code change (docs, etc) - [ ] None of the above: (explain here) ## Test Plan Updated NamespacesTests.java to include: - `createNamespace()` call - `prefix` and `totalCount`
1 parent 3e3c086 commit c01528e

File tree

5 files changed

+261
-9
lines changed

5 files changed

+261
-9
lines changed

‎README.md‎

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -697,6 +697,36 @@ List<Float> values = Arrays.asList(1F, 2F, 3F);
697697
UpdateResponse updateResponse = index.update("v1", values, "example-namespace");
698698
```
699699

700+
## Create namespace
701+
702+
The following example shows how to create a namespace.
703+
704+
```java
705+
import io.pinecone.clients.AsyncIndex;
706+
import io.pinecone.clients.Index;
707+
import io.pinecone.clients.Pinecone;
708+
import io.pinecone.proto.MetadataFieldProperties;
709+
import io.pinecone.proto.MetadataSchema;
710+
import io.pinecone.proto.NamespaceDescription;
711+
712+
import java.util.concurrent.ExecutionException;
713+
...
714+
715+
String indexName = "PINECONE_INDEX_NAME";
716+
Pinecone pinecone = new Pinecone.Builder("PINECONE_API_KEY").build();
717+
Index index = pinecone.getIndexConnection(indexName);
718+
719+
// create a namespace
720+
NamespaceDescription namespaceDescription = index.createNamespace("some-namespace");
721+
722+
// create a namespace with metadata schema
723+
MetadataSchema schema = MetadataSchema.newBuilder()
724+
.putFields("genre", MetadataFieldProperties.newBuilder().setFilterable(true).build())
725+
.putFields("year", MetadataFieldProperties.newBuilder().setFilterable(true).build())
726+
.build();
727+
NamespaceDescription namespaceWithSchema = index.createNamespace("some-namespace", schema);
728+
```
729+
700730
## List namespaces
701731

702732
The following example shows various methods to list namespaces.
@@ -722,6 +752,19 @@ listNamespacesResponse = index.listNamespaces("some-pagination-token");
722752

723753
// list namespaces with pagination token and a custom limit of 5
724754
listNamespacesResponse = index.listNamespaces("some-pagination-token", 5);
755+
756+
// The totalCount field returns the total number of namespaces in the index
757+
// When prefix filtering is used, it returns the count of namespaces matching the prefix
758+
int totalCount = listNamespacesResponse.getTotalCount();
759+
760+
// list namespaces with prefix filtering
761+
// Prefix filtering allows you to filter namespaces that start with a specific prefix
762+
listNamespacesResponse = index.listNamespaces("test-", null, 10);
763+
totalCount = listNamespacesResponse.getTotalCount(); // Total count of namespaces matching "test-" prefix
764+
765+
// list namespaces with prefix, pagination token, and limit
766+
listNamespacesResponse = index.listNamespaces("test-", "some-pagination-token", 10);
767+
totalCount = listNamespacesResponse.getTotalCount();
725768
```
726769

727770
## Describe namespace

‎src/integration/java/io/pinecone/integration/dataPlane/NamespacesTest.java‎

Lines changed: 62 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -5,13 +5,16 @@
55
import io.pinecone.helpers.RandomStringBuilder;
66
import io.pinecone.helpers.TestResourcesManager;
77
import io.pinecone.proto.ListNamespacesResponse;
8+
import io.pinecone.proto.NamespaceDescription;
89
import org.junit.jupiter.api.AfterAll;
910
import org.junit.jupiter.api.Test;
1011

1112
import java.util.concurrent.ExecutionException;
1213

1314
import static io.pinecone.helpers.BuildUpsertRequest.generateVectorValuesByDimension;
1415
import static org.junit.jupiter.api.Assertions.assertEquals;
16+
import static org.junit.jupiter.api.Assertions.assertNotNull;
17+
import static org.junit.jupiter.api.Assertions.assertTrue;
1518

1619
public class NamespacesTest {
1720
private static final TestResourcesManager indexManager = TestResourcesManager.getInstance();
@@ -27,54 +30,104 @@ public static void cleanUp() {
2730

2831
@Test
2932
public void namespacesSyncTest() throws InterruptedException {
30-
String[] namespaces = new String[3];
33+
String[] namespaces = new String[4];
3134
index = indexManager.getOrCreateServerlessIndexConnection();
3235
ListNamespacesResponse listNamespacesResponse = index.listNamespaces();
3336
int namespaceCount = listNamespacesResponse.getNamespacesCount();
3437

35-
for(int i=0; i<3; i++) {
38+
// Create namespace explicitly using createNamespace
39+
namespaces[0] = RandomStringBuilder.build("namespace-", 3);
40+
NamespaceDescription createdNamespace = index.createNamespace(namespaces[0]);
41+
assertNotNull(createdNamespace);
42+
assertEquals(namespaces[0], createdNamespace.getName());
43+
44+
// Create namespaces implicitly by upserting vectors
45+
for (int i=1; i<4; i++) {
3646
namespaces[i] = RandomStringBuilder.build("namespace-", 3);
3747
index.upsert("v"+i, generateVectorValuesByDimension(dimension), namespaces[i]);
3848
}
3949

4050
// wait for vectors to be upserted
4151
Thread.sleep(5000);
4252
listNamespacesResponse = index.listNamespaces();
43-
assertEquals(listNamespacesResponse.getNamespacesCount(), namespaceCount + 3);
53+
assertEquals(listNamespacesResponse.getNamespacesCount(), namespaceCount + 4);
4454

4555
index.describeNamespace(namespaces[0]);
4656
index.deleteNamespace(namespaces[0]);
4757

4858
// wait for namespace to be deleted
4959
Thread.sleep(3000);
5060
listNamespacesResponse = index.listNamespaces();
51-
assertEquals(listNamespacesResponse.getNamespacesCount(), namespaceCount + 2);
61+
assertEquals(listNamespacesResponse.getNamespacesCount(), namespaceCount + 3);
62+
63+
// Test prefix filtering and total count
64+
String prefix = "namespace-";
65+
ListNamespacesResponse prefixResponse = index.listNamespaces(prefix, null, 100);
66+
assertNotNull(prefixResponse);
67+
assertTrue(prefixResponse.getTotalCount() >= 3, "totalCount should be at least 3");
68+
assertTrue(prefixResponse.getNamespacesCount() >= 3, "Should return at least 3 namespaces with prefix");
69+
70+
// Verify all returned namespaces start with the prefix
71+
for (int i = 0; i < prefixResponse.getNamespacesCount(); i++) {
72+
String namespaceName = prefixResponse.getNamespaces(i).getName();
73+
assertTrue(namespaceName.startsWith(prefix),
74+
"Namespace " + namespaceName + " should start with prefix " + prefix);
75+
}
76+
77+
// Verify totalCount is at least the number of namespaces returned
78+
assertTrue(prefixResponse.getTotalCount() >= prefixResponse.getNamespacesCount(),
79+
"totalCount should be at least equal to the number of namespaces returned");
5280
}
5381

5482
@Test
5583
public void namespacesAsyncTest() throws InterruptedException, ExecutionException {
56-
String[] namespaces = new String[3];
84+
String[] namespaces = new String[4];
5785
asyncIndex = indexManager.getOrCreateServerlessAsyncIndexConnection();
5886

5987
ListNamespacesResponse listNamespacesResponse = asyncIndex.listNamespaces().get();
6088
int namespaceCount = listNamespacesResponse.getNamespacesCount();
6189

62-
for(int i=0; i<3; i++) {
90+
// Create namespace explicitly using createNamespace
91+
namespaces[0] = RandomStringBuilder.build("namespace-", 3);
92+
NamespaceDescription createdNamespace = asyncIndex.createNamespace(namespaces[0]).get();
93+
assertNotNull(createdNamespace);
94+
assertEquals(namespaces[0], createdNamespace.getName());
95+
96+
// Create namespaces implicitly by upserting vectors
97+
for (int i=1; i<4; i++) {
6398
namespaces[i] = RandomStringBuilder.build("namespace-", 3);
6499
asyncIndex.upsert("v"+i, generateVectorValuesByDimension(dimension), namespaces[i]);
65100
}
66101

67102
// wait for vectors to be upserted
68103
Thread.sleep(5000);
69104
listNamespacesResponse = asyncIndex.listNamespaces().get();
70-
assertEquals(listNamespacesResponse.getNamespacesCount(), namespaceCount + 3);
105+
assertEquals(listNamespacesResponse.getNamespacesCount(), namespaceCount + 4);
71106

72107
asyncIndex.describeNamespace(namespaces[0]);
73108
asyncIndex.deleteNamespace(namespaces[0]);
74109

75110
// wait for namespace to be deleted
76111
Thread.sleep(3000);
77112
listNamespacesResponse = asyncIndex.listNamespaces().get();
78-
assertEquals(listNamespacesResponse.getNamespacesCount(), namespaceCount + 2);
113+
assertEquals(listNamespacesResponse.getNamespacesCount(), namespaceCount + 3);
114+
115+
// Test prefix filtering and total count
116+
String prefix = "namespace-";
117+
ListNamespacesResponse prefixResponse = asyncIndex.listNamespaces(prefix, null, 100).get();
118+
assertNotNull(prefixResponse);
119+
assertTrue(prefixResponse.getTotalCount() >= 3, "totalCount should be at least 3");
120+
assertTrue(prefixResponse.getNamespacesCount() >= 3, "Should return at least 3 namespaces with prefix");
121+
122+
// Verify all returned namespaces start with the prefix
123+
for (int i = 0; i < prefixResponse.getNamespacesCount(); i++) {
124+
String namespaceName = prefixResponse.getNamespaces(i).getName();
125+
assertTrue(namespaceName.startsWith(prefix),
126+
"Namespace " + namespaceName + " should start with prefix " + prefix);
127+
}
128+
129+
// Verify totalCount is at least the number of namespaces returned
130+
assertTrue(prefixResponse.getTotalCount() >= prefixResponse.getNamespacesCount(),
131+
"totalCount should be at least equal to the number of namespaces returned");
79132
}
80-
}
133+
}

‎src/main/java/io/pinecone/clients/AsyncIndex.java‎

Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,11 +9,13 @@
99
import io.pinecone.configs.PineconeConnection;
1010
import io.pinecone.exceptions.PineconeValidationException;
1111
import io.pinecone.proto.*;
12+
import io.pinecone.proto.CreateNamespaceRequest;
1213
import io.pinecone.proto.DeleteRequest;
1314
import io.pinecone.proto.DescribeIndexStatsRequest;
1415
import io.pinecone.proto.FetchResponse;
1516
import io.pinecone.proto.ListNamespacesResponse;
1617
import io.pinecone.proto.ListResponse;
18+
import io.pinecone.proto.MetadataSchema;
1719
import io.pinecone.proto.NamespaceDescription;
1820
import io.pinecone.proto.QueryRequest;
1921
import io.pinecone.proto.QueryResponse;
@@ -1124,6 +1126,66 @@ public ListenableFuture<ListNamespacesResponse> listNamespaces(String pagination
11241126
return asyncStub.listNamespaces(listNamespacesRequest.build());
11251127
}
11261128

1129+
/**
1130+
* <pre>
1131+
* Get list of all namespaces within an index with optional prefix filtering, pagination token, and limit.
1132+
* @param prefix The prefix to filter namespaces by. Only namespaces starting with this prefix will be returned.
1133+
* If null, no prefix filtering is applied.
1134+
* @param paginationToken The token to paginate through the list of namespaces. If null, it'll be ignored.
1135+
* @param limit The maximum number of namespaces you want to retrieve.
1136+
* @return {@link ListenableFuture<ListNamespacesResponse>} The response for the list namespace operation. The totalCount field
1137+
* indicates the total number of namespaces matching the prefix (if provided).
1138+
* </pre>
1139+
*/
1140+
@Override
1141+
public ListenableFuture<ListNamespacesResponse> listNamespaces(String prefix, String paginationToken, int limit) {
1142+
ListNamespacesRequest.Builder listNamespacesRequest = ListNamespacesRequest
1143+
.newBuilder()
1144+
.setLimit(limit);
1145+
if(prefix != null && !prefix.isEmpty()) {
1146+
listNamespacesRequest.setPrefix(prefix);
1147+
}
1148+
if(paginationToken != null) {
1149+
listNamespacesRequest.setPaginationToken(paginationToken);
1150+
}
1151+
return asyncStub.listNamespaces(listNamespacesRequest.build());
1152+
}
1153+
1154+
/**
1155+
* <pre>
1156+
* Create a namespace within an index.
1157+
* @param name The name of the namespace to create.
1158+
* @return {@link ListenableFuture<NamespaceDescription>} The response for the create namespace operation.
1159+
* </pre>
1160+
*/
1161+
@Override
1162+
public ListenableFuture<NamespaceDescription> createNamespace(String name) {
1163+
CreateNamespaceRequest createNamespaceRequest = CreateNamespaceRequest
1164+
.newBuilder()
1165+
.setName(name)
1166+
.build();
1167+
return asyncStub.createNamespace(createNamespaceRequest);
1168+
}
1169+
1170+
/**
1171+
* <pre>
1172+
* Create a namespace within an index with a metadata schema.
1173+
* @param name The name of the namespace to create.
1174+
* @param schema The metadata schema for the namespace.
1175+
* @return {@link ListenableFuture<NamespaceDescription>} The response for the create namespace operation.
1176+
* </pre>
1177+
*/
1178+
@Override
1179+
public ListenableFuture<NamespaceDescription> createNamespace(String name, MetadataSchema schema) {
1180+
CreateNamespaceRequest.Builder builder = CreateNamespaceRequest
1181+
.newBuilder()
1182+
.setName(name);
1183+
if (schema != null) {
1184+
builder.setSchema(schema);
1185+
}
1186+
return asyncStub.createNamespace(builder.build());
1187+
}
1188+
11271189
/**
11281190
* <pre>
11291191
* Describe a namespace within an index, showing the vector count within the namespace.

‎src/main/java/io/pinecone/clients/Index.java‎

Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,11 +6,13 @@
66
import io.pinecone.configs.PineconeConnection;
77
import io.pinecone.exceptions.PineconeValidationException;
88
import io.pinecone.proto.*;
9+
import io.pinecone.proto.CreateNamespaceRequest;
910
import io.pinecone.proto.DeleteRequest;
1011
import io.pinecone.proto.DescribeIndexStatsRequest;
1112
import io.pinecone.proto.FetchResponse;
1213
import io.pinecone.proto.ListNamespacesResponse;
1314
import io.pinecone.proto.ListResponse;
15+
import io.pinecone.proto.MetadataSchema;
1416
import io.pinecone.proto.NamespaceDescription;
1517
import io.pinecone.proto.QueryRequest;
1618
import io.pinecone.proto.UpdateRequest;
@@ -1034,6 +1036,66 @@ public ListNamespacesResponse listNamespaces(String paginationToken, int limit)
10341036
return blockingStub.listNamespaces(listNamespacesRequest.build());
10351037
}
10361038

1039+
/**
1040+
* <pre>
1041+
* Get list of all namespaces within an index with optional prefix filtering, pagination token, and limit.
1042+
* @param prefix The prefix to filter namespaces by. Only namespaces starting with this prefix will be returned.
1043+
* If null, no prefix filtering is applied.
1044+
* @param paginationToken The token to paginate through the list of namespaces. If null, it'll be ignored.
1045+
* @param limit The maximum number of namespaces you want to retrieve.
1046+
* @return {@link ListNamespacesResponse} The response for the list namespace operation. The totalCount field
1047+
* indicates the total number of namespaces matching the prefix (if provided).
1048+
* </pre>
1049+
*/
1050+
@Override
1051+
public ListNamespacesResponse listNamespaces(String prefix, String paginationToken, int limit) {
1052+
ListNamespacesRequest.Builder listNamespacesRequest = ListNamespacesRequest
1053+
.newBuilder()
1054+
.setLimit(limit);
1055+
if(prefix != null && !prefix.isEmpty()) {
1056+
listNamespacesRequest.setPrefix(prefix);
1057+
}
1058+
if(paginationToken != null) {
1059+
listNamespacesRequest.setPaginationToken(paginationToken);
1060+
}
1061+
return blockingStub.listNamespaces(listNamespacesRequest.build());
1062+
}
1063+
1064+
/**
1065+
* <pre>
1066+
* Create a namespace within an index.
1067+
* @param name The name of the namespace to create.
1068+
* @return {@link NamespaceDescription} The response for the create namespace operation.
1069+
* </pre>
1070+
*/
1071+
@Override
1072+
public NamespaceDescription createNamespace(String name) {
1073+
CreateNamespaceRequest createNamespaceRequest = CreateNamespaceRequest
1074+
.newBuilder()
1075+
.setName(name)
1076+
.build();
1077+
return blockingStub.createNamespace(createNamespaceRequest);
1078+
}
1079+
1080+
/**
1081+
* <pre>
1082+
* Create a namespace within an index with a metadata schema.
1083+
* @param name The name of the namespace to create.
1084+
* @param schema The metadata schema for the namespace.
1085+
* @return {@link NamespaceDescription} The response for the create namespace operation.
1086+
* </pre>
1087+
*/
1088+
@Override
1089+
public NamespaceDescription createNamespace(String name, MetadataSchema schema) {
1090+
CreateNamespaceRequest.Builder builder = CreateNamespaceRequest
1091+
.newBuilder()
1092+
.setName(name);
1093+
if (schema != null) {
1094+
builder.setSchema(schema);
1095+
}
1096+
return blockingStub.createNamespace(builder.build());
1097+
}
1098+
10371099
/**
10381100
* <pre>
10391101
* Describe a namespace within an index, showing the vector count within the namespace.

‎src/main/java/io/pinecone/commons/IndexInterface.java‎

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -854,6 +854,38 @@ default void validateListEndpointParameters(String namespace, String prefix, Str
854854
*/
855855
A listNamespaces(String paginationToken, int limit);
856856

857+
/**
858+
* <pre>
859+
* Get list of all namespaces within an index with optional prefix filtering, pagination token, and limit.
860+
* @param prefix The prefix to filter namespaces by. Only namespaces starting with this prefix will be returned.
861+
* If null, no prefix filtering is applied.
862+
* @param paginationToken The token to paginate through the list of namespaces. If null, it'll be ignored.
863+
* @param limit The maximum number of namespaces you want to retrieve.
864+
* @return {@link ListNamespacesResponse} The response for the list namespace operation. The totalCount field
865+
* indicates the total number of namespaces matching the prefix (if provided).
866+
* </pre>
867+
*/
868+
A listNamespaces(String prefix, String paginationToken, int limit);
869+
870+
/**
871+
* <pre>
872+
* Create a namespace within an index.
873+
* @param name The name of the namespace to create.
874+
* @return {@link NamespaceDescription} The response for the create namespace operation.
875+
* </pre>
876+
*/
877+
B createNamespace(String name);
878+
879+
/**
880+
* <pre>
881+
* Create a namespace within an index with a metadata schema.
882+
* @param name The name of the namespace to create.
883+
* @param schema The metadata schema for the namespace.
884+
* @return {@link NamespaceDescription} The response for the create namespace operation.
885+
* </pre>
886+
*/
887+
B createNamespace(String name, io.pinecone.proto.MetadataSchema schema);
888+
857889
/**
858890
* <pre>
859891
* Describe a namespace within an index, showing the vector count within the namespace.

0 commit comments

Comments
 (0)