Spark UI Repr is Too Verbose

[matan129]

Hi, when using Elasticsearch with Spark, the ElasticsearchRelation string repr is super verbose. It prints out all the settings of the ES Connector, including the query body, fields and other parameters:

Note the horizontal scrollbar - in this case, I used quite a long query and it makes the Spark plan basically unreadable since I can't fit it on one screen.

Can you please patch the relevant part of ElasticsearchRelation? I wasn't sure if this is just the .toString() method, or something else.

Thanks!

[masseyke]

Hi @matan129. What do you think would be useful (if anything) to have here?

[matan129]

Maybe just the index name, or just Scan ElasticsearchRelation and that's it.

Another option is to add a setting that limits the number of characters in the description, but that would be an overkill IMHO.