Unable to commit index to AWS OpenSearch #115
Description
I set up a domain named "mysearch" on AWS OpenSearch. Its end point is https://search-mysearch-abcd1234.us-east-1.es.amazonaws.com/ and it's set to be public accessible. I installed Norconex http collector 3.0.0 and ElasticSearch committer 5.0.0 on my local desktop and tried to index some HTML pages and commit it to my OpenSearch domain. The committer configuration is like the following:
<committer class="ElasticsearchCommitter">
<nodes>https://search-mysearch-abcd1234.us-east-1.es.amazonaws.com/</nodes>
<indexName>genetics</indexName>
<credentials>
<username>myusername</username>
<password>mypassword</password>
</credentials>
</committer>
When I started indexing, I got an error saying "ERROR ElasticsearchCommitter - Failure occured on node: "null". Check node logs." I am not sure where the log is.
To test where the problem is, I started an ElasticSearch instance on my local desktop and changed the value to http://127.0.0.1:9200 in the config file, and it successfully index the HTML pages to my local ElasticSearch instance. So it proves that my code and config file are fine.
I also ran a command on my local desktop to index a JSON file and commit it to my AWS OpenSearch instance like this:
curl -H "Content-Type:application/json" -XPOST -u "username:password" "https://search-mysearch-abcd1234.us-east-1.es.amazonaws.com/_bulk" --data-binary "@mydata.json".
This also successfully index the JSON contents to my OpenSearch instance, which proves that there's no connection problem from my desktop to my OpenSearch instance.
So I suspect it's the setting in the config file that's causing the problem. Can you please advise if using the OpenSearch endpoint in the <nodes> element is incorrect. If so, what should I change it to?
Many thanks!
CY