Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SOLR-15751: Modify v2 GET /collections/collName to support full ColStatus response #2912

Open
wants to merge 15 commits into
base: main
Choose a base branch
from
Open
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.solr.client.api.endpoint;

import io.swagger.v3.oas.annotations.Operation;
import io.swagger.v3.oas.annotations.Parameter;
import jakarta.ws.rs.GET;
import jakarta.ws.rs.Path;
import jakarta.ws.rs.PathParam;
import jakarta.ws.rs.QueryParam;
import org.apache.solr.client.api.model.CollectionStatusResponse;

/**
* V2 API definition for fetching collection metadata
*
* <p>This API (GET /v2/collections/collectionName) is analogous to the v1
* /admin/collections?action=COLSTATUS command.
*/
@Path("/collections/{collectionName}")
public interface CollectionStatusApi {

// TODO Query parameters currently match those offered by the v1
// /admin/collections?action=COLSTATUS. Should param names be updated/clarified?
@GET
@Operation(
summary = "Fetches metadata about the specified collection",
tags = {"collections"})
CollectionStatusResponse getCollectionStatus(
@Parameter(description = "The name of the collection return metadata for", required = true)
@PathParam("collectionName")
String collectionName,
@Parameter(description = SegmentsApi.CORE_INFO_PARAM_DESC) @QueryParam("coreInfo")
Boolean coreInfo,
@Parameter(
description =
"Boolean flag to include metadata and statistics about the segments used by each shard leader. Implicitly set to true by 'fieldInfo' and 'sizeInfo'")
@QueryParam("segments")
Boolean segments,
@Parameter(
description =
SegmentsApi.FIELD_INFO_PARAM_DESC
+ " Implicitly sets the 'segments' flag to 'true'")
@QueryParam("fieldInfo")
Boolean fieldInfo,
@Parameter(description = SegmentsApi.RAW_SIZE_PARAM_DESC) @QueryParam("rawSize")
Boolean rawSize,
@Parameter(description = SegmentsApi.RAW_SIZE_SUMMARY_DESC) @QueryParam("rawSizeSummary")
Boolean rawSizeSummary,
@Parameter(description = SegmentsApi.RAW_SIZE_DETAILS_DESC) @QueryParam("rawSizeDetails")
Boolean rawSizeDetails,
@Parameter(description = SegmentsApi.RAW_SIZE_SAMPLING_PERCENT_DESC)
@QueryParam("rawSizeSamplingPercent")
Float rawSizeSamplingPercent,
@Parameter(
description =
SegmentsApi.SIZE_INFO_PARAM_DESC
+ ". Implicitly sets the 'segment' flag to 'true'")
@QueryParam("sizeInfo")
Boolean sizeInfo)
throws Exception;
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a shame to repeat this stuff. Is it possible to maybe subclass the API so that the CollectionStatusApi inherits SegmentsApi? If the only gotcha is a desire to document that some booleans require others then I'd rather we simply have the API return a 400 and tell the user that if you enable X then you must also enable Y. Quite fair & helpful IMO.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fwiw, we've already got some reuse here: e.g. these SegmentsApi constants get re-used in CollectionStatusApi as well.

The main thing that prevents us from doing something more with inheritance, etc. is that the method signatures for CollectionStatusApi/SegmentsApi aren't the same. The CollectionStatusApi method takes two parameters ('collectionName' and 'segments') that have no SegmentApi equivalent.

Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.solr.client.api.endpoint;

import io.swagger.v3.oas.annotations.Operation;
import io.swagger.v3.oas.annotations.Parameter;
import jakarta.ws.rs.GET;
import jakarta.ws.rs.Path;
import jakarta.ws.rs.QueryParam;
import org.apache.solr.client.api.model.GetSegmentDataResponse;
import org.apache.solr.client.api.util.CoreApiParameters;

/**
* V2 API definition for fetching metadata about a core's segments
*
* <p>This API (GET /v2/cores/coreName/segments) is analogous to the v1
* /solr/coreName/admin/segments API
*/
@Path("/cores/{coreName}/segments")
public interface SegmentsApi {

String CORE_INFO_PARAM_DESC =
"Boolean flag to include metadata (e.g. index an data directories, IndexWriter configuration, etc.) about each shard leader's core";
String FIELD_INFO_PARAM_DESC =
"Boolean flag to include statistics about the indexed fields present on each shard leader.";
String RAW_SIZE_PARAM_DESC =
"Boolean flag to include simple estimates of the disk size taken up by each field (e.g. \"id\", \"_version_\") and by each index data structure (e.g. 'storedFields', 'docValues_numeric').";
String RAW_SIZE_SUMMARY_DESC =
"Boolean flag to include more involved estimates of the disk size taken up by index data structures, on a per-field basis (e.g. how much data does the \"id\" field contribute to 'storedField' index files). More detail than 'rawSize', less detail than 'rawSizeDetails'.";
String RAW_SIZE_DETAILS_DESC =
"Boolean flag to include detailed statistics about the disk size taken up by various fields and data structures. More detail than 'rawSize' and 'rawSizeSummary'.";
String RAW_SIZE_SAMPLING_PERCENT_DESC =
"Percentage (between 0 and 100) of data to read when estimating index size and statistics. Defaults to 5.0 (i.e. 5%).";
String SIZE_INFO_PARAM_DESC =
"Boolean flag to include information about the largest index files for each Lucene segment.";

@GET
@CoreApiParameters
@Operation(
summary = "Fetches metadata about the segments in use by the specified core",
tags = {"segments"})
GetSegmentDataResponse getSegmentData(
@Parameter(description = CORE_INFO_PARAM_DESC) @QueryParam("coreInfo") Boolean coreInfo,
@Parameter(description = FIELD_INFO_PARAM_DESC) @QueryParam("fieldInfo") Boolean fieldInfo,
@Parameter(description = RAW_SIZE_PARAM_DESC) @QueryParam("rawSize") Boolean rawSize,
@Parameter(description = RAW_SIZE_SUMMARY_DESC) @QueryParam("rawSizeSummary")
Boolean rawSizeSummary,
@Parameter(description = RAW_SIZE_DETAILS_DESC) @QueryParam("rawSizeDetails")
Boolean rawSizeDetails,
@Parameter(description = RAW_SIZE_SAMPLING_PERCENT_DESC) @QueryParam("rawSizeSamplingPercent")
Float rawSizeSamplingPercent,
@Parameter(description = SIZE_INFO_PARAM_DESC) @QueryParam("sizeInfo") Boolean sizeInfo)
throws Exception;
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,141 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.solr.client.api.model;

import com.fasterxml.jackson.annotation.JsonAnyGetter;
import com.fasterxml.jackson.annotation.JsonAnySetter;
import com.fasterxml.jackson.annotation.JsonProperty;
import java.util.HashMap;
import java.util.List;
import java.util.Map;

/**
* Response of the CollectionStatusApi.getCollectionStatus() API
*
* <p>Note that the corresponding v1 API has a slightly different response format. Users should not
* attempt to convert a v1 response into this type.
*/
public class CollectionStatusResponse extends SolrJerseyResponse {
gerlowskija marked this conversation as resolved.
Show resolved Hide resolved

@JsonProperty public String name;
@JsonProperty public Integer znodeVersion;
// TODO - consider 'Instant' once SOLR-17608 is finished
@JsonProperty public Long creationTimeMillis;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use Instant and drop "Millis"?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can change the variable name and probably the type on the deserialized-Java side.

But we'll need to keep the same serialization-time prop name and value to avoid backcompat issues.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will Date work in the interim?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, Date can work until SOLR-17608 gets sorted out. If we think "Instant" is generally better, I can switch to 'Date', but add in a TODO comment with the JIRA ticket, so we can find this later on...

@JsonProperty public CollectionMetadata properties;
@JsonProperty public Integer activeShards;
@JsonProperty public Integer inactiveShards;
@JsonProperty public List<String> schemaNonCompliant;

@JsonProperty public Map<String, ShardMetadata> shards;

// Always present in response
public static class CollectionMetadata {
@JsonProperty public String configName;
@JsonProperty public Integer nrtReplicas;
@JsonProperty public Integer pullReplicas;
@JsonProperty public Integer tlogReplicas;
@JsonProperty public Map<String, String> router;
@JsonProperty public Integer replicationFactor;

private Map<String, Object> unknownFields = new HashMap<>();

@JsonAnyGetter
public Map<String, Object> unknownProperties() {
return unknownFields;
}

@JsonAnySetter
public void setUnknownProperty(String field, Object value) {
unknownFields.put(field, value);
}
}

// Always present in response
public static class ShardMetadata {
@JsonProperty public String state; // TODO Make this an enum?
@JsonProperty public String range;
@JsonProperty public ReplicaSummary replicas;
@JsonProperty public LeaderSummary leader;
}

// Always present in response
public static class ReplicaSummary {
@JsonProperty public Integer total;
@JsonProperty public Integer active;
@JsonProperty public Integer down;
@JsonProperty public Integer recovering;

@JsonProperty("recovery_failed")
public Integer recoveryFailed;
}

// Always present in response unless otherwise specified
public static class LeaderSummary {
@JsonProperty public String coreNode;
@JsonProperty public String core;
@JsonProperty public Boolean leader;

@JsonProperty("node_name")
public String nodeName;

@JsonProperty("base_url")
public String baseUrl;

@JsonProperty public String state; // TODO Make this an enum?
@JsonProperty public String type; // TODO Make this an enum?

@JsonProperty("force_set_state")
public Boolean forceSetState;
gerlowskija marked this conversation as resolved.
Show resolved Hide resolved

// Present with coreInfo=true || sizeInfo=true unless otherwise specified
@JsonProperty public SegmentInfo segInfos;

private Map<String, Object> unknownFields = new HashMap<>();

@JsonAnyGetter
public Map<String, Object> unknownProperties() {
return unknownFields;
}

@JsonAnySetter
public void setUnknownProperty(String field, Object value) {
unknownFields.put(field, value);
}
}

// Present with segments=true || coreInfo=true || sizeInfo=true || fieldInfo=true unless otherwise
// specified

/**
* Same properties as {@link GetSegmentDataResponse}, but uses a different class to avoid
* inheriting "responseHeader", etc.
*/
public static class SegmentInfo {
@JsonProperty public GetSegmentDataResponse.SegmentSummary info;

@JsonProperty public Map<String, Object> runningMerges;

// Present with segments=true || sizeInfo=true || fieldInfo=true
@JsonProperty public Map<String, GetSegmentDataResponse.SingleSegmentData> segments;

// Present with rawSize=true
@JsonProperty public GetSegmentDataResponse.RawSize rawSize;

// Present only with fieldInfo=true
@JsonProperty public List<String> fieldInfoLegend;
}
}
Loading
Loading