Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ZEPPELIN-6131] Update Interpreter to Store Values as Integers When Applicable #4878

Merged

Conversation

ParkGyeongTae
Copy link
Contributor

@ParkGyeongTae ParkGyeongTae commented Oct 18, 2024

What is this PR for?

There was an issue where the port was treated as a string and included .0 when modifying the port of the Elasticsearch interpreter.

What type of PR is it?

Bug Fix

Todos

  • - Modified HttpBasedClient.java file

What is the Jira issue?

How should this be tested?

  • Zeppelin project build and run
./mvnw clean package -DskipTests
  • Add a test index and test data to Elasticsearch
# Create an index
curl -X PUT "http://localhost:9200/customer?pretty"

# Create a document
curl -X \
PUT "localhost:9200/customer/_doc/1" \
-H 'Content-Type: application/json' \
-d'
{
  "field_1": "value_1"
}'
  • Modify the Elasticsearch interpreter type and port
    elasticsearch.client.type: transport -> http
    elasticsearch.port: 9300 -> 9200

  • Previous configuration

image
  • New configuration
image
  • Create a Zeppelin notebook and execute a paragraph
%elasticsearch
get /customer/_doc/1
  • Result before modification - Error occurred
image
org.apache.zeppelin.interpreter.InterpreterException: java.lang.NumberFormatException: For input string: "9200.0"
	at org.apache.zeppelin.interpreter.LazyOpenInterpreter.open(LazyOpenInterpreter.java:77)
	at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:808)
	at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:716)
	at org.apache.zeppelin.scheduler.Job.run(Job.java:187)
	at org.apache.zeppelin.scheduler.AbstractScheduler.runJob(AbstractScheduler.java:136)
	at org.apache.zeppelin.scheduler.FIFOScheduler.lambda$runJobInScheduler$0(FIFOScheduler.java:42)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.lang.NumberFormatException: For input string: "9200.0"
	at java.base/java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
	at java.base/java.lang.Integer.parseInt(Integer.java:652)
	at java.base/java.lang.Integer.parseInt(Integer.java:770)
	at org.apache.zeppelin.elasticsearch.client.HttpBasedClient.<init>(HttpBasedClient.java:67)
	at org.apache.zeppelin.elasticsearch.ElasticsearchInterpreter.open(ElasticsearchInterpreter.java:136)
	at org.apache.zeppelin.interpreter.LazyOpenInterpreter.open(LazyOpenInterpreter.java:71)
	... 8 more
  • Result after modification
image

Screenshots (if appropriate)

  • Refer to the above content

Questions:

  • Does the license files need to update? No.
  • Is there breaking changes for older versions? No.
  • Does this needs documentation? No.

@Reamer
Copy link
Contributor

Reamer commented Oct 18, 2024

Do you know where the .0 comes from? We should rather switch off this behavior than fix the error later.

@Reamer
Copy link
Contributor

Reamer commented Oct 31, 2024

I like this approach. Can you write a unit test that tests the behavior with different combinations?

@ParkGyeongTae
Copy link
Contributor Author

@Reamer
Thank you for your guidance. When modifying interpreters in the Zeppelin Web UI, I discovered that integers were being saved as floats in the interpreter.json file. To fix this, I modified it to convert floats to integers before saving, where possible. Additionally, I added test code.

Copy link
Contributor

@Reamer Reamer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM except one minor issue.

Copy link
Contributor

@Reamer Reamer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, except the Pull Request title. We should change the title to match the actual change.

My suggestion, but I'm not so happy with it.
Preference for integers instead of floating point numbers for interpreter update requests.

EDIT: Of course, the CI pipeline still has to run through successfully.

@ParkGyeongTae ParkGyeongTae changed the title [ZEPPELIN-6131] Error occurred when modifying Elasticsearch interpreter port [ZEPPELIN-6131] Update Interpreter to Store Values as Integers When Applicable Nov 12, 2024
@ParkGyeongTae
Copy link
Contributor Author

@Reamer I have updated the PR title to "Update Interpreter to Store Values as Integers When Applicable."

@Reamer Reamer merged commit 4a7fdb1 into apache:master Nov 12, 2024
17 checks passed
asfgit pushed a commit that referenced this pull request Nov 12, 2024
…pplicable

### What is this PR for?
There was an issue where the port was treated as a string and included `.0` when modifying the port of the Elasticsearch interpreter.

### What type of PR is it?
Bug Fix

### Todos
* [x] - Modified HttpBasedClient.java file

### What is the Jira issue?
* https://issues.apache.org/jira/projects/ZEPPELIN/issues/ZEPPELIN-6131

### How should this be tested?
* Zeppelin project build and run
```bash
./mvnw clean package -DskipTests
```

* Add a test index and test data to Elasticsearch
```bash
# Create an index
curl -X PUT "http://localhost:9200/customer?pretty"

# Create a document
curl -X \
PUT "localhost:9200/customer/_doc/1" \
-H 'Content-Type: application/json' \
-d'
{
  "field_1": "value_1"
}'
```

* Modify the Elasticsearch interpreter type and port
elasticsearch.client.type: transport -> http
elasticsearch.port: 9300 -> 9200

* Previous configuration
<img width="1228" alt="image" src="https://github.com/user-attachments/assets/aab69016-b982-4817-8d72-68186fdd45b8">

* New configuration
<img width="1219" alt="image" src="https://github.com/user-attachments/assets/a9c44503-1ed1-43c0-89f9-19b92a1ac1e0">

* Create a Zeppelin notebook and execute a paragraph
```bash
%elasticsearch
get /customer/_doc/1
```

* Result before modification - Error occurred
<img width="942" alt="image" src="https://github.com/user-attachments/assets/00d5db2f-e1dd-46bf-96d9-eb2eb1d88072">

```bash
org.apache.zeppelin.interpreter.InterpreterException: java.lang.NumberFormatException: For input string: "9200.0"
	at org.apache.zeppelin.interpreter.LazyOpenInterpreter.open(LazyOpenInterpreter.java:77)
	at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:808)
	at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:716)
	at org.apache.zeppelin.scheduler.Job.run(Job.java:187)
	at org.apache.zeppelin.scheduler.AbstractScheduler.runJob(AbstractScheduler.java:136)
	at org.apache.zeppelin.scheduler.FIFOScheduler.lambda$runJobInScheduler$0(FIFOScheduler.java:42)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.lang.NumberFormatException: For input string: "9200.0"
	at java.base/java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
	at java.base/java.lang.Integer.parseInt(Integer.java:652)
	at java.base/java.lang.Integer.parseInt(Integer.java:770)
	at org.apache.zeppelin.elasticsearch.client.HttpBasedClient.<init>(HttpBasedClient.java:67)
	at org.apache.zeppelin.elasticsearch.ElasticsearchInterpreter.open(ElasticsearchInterpreter.java:136)
	at org.apache.zeppelin.interpreter.LazyOpenInterpreter.open(LazyOpenInterpreter.java:71)
	... 8 more
```

* Result after modification
<img width="500" alt="image" src="https://github.com/user-attachments/assets/2634216c-ed42-4a1e-8c6d-d3bb378365fa">

### Screenshots (if appropriate)
* Refer to the above content

### Questions:
* Does the license files need to update? No.
* Is there breaking changes for older versions? No.
* Does this needs documentation? No.


Closes #4878 from ParkGyeongTae/fix-elasticsearch-port-string-handling.

Signed-off-by: Philipp Dallig <[email protected]>
@Reamer
Copy link
Contributor

Reamer commented Nov 12, 2024

Merged into master/branch-0.12

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants