Re: Solr with HDFS configuration example running in production/dev

Hi

I would not go down this road. What is the use case?  Is this really the sol=
ution?

Go read all the relevant docs and configuration provided by Cloudera/HortonW=
orks and everything else related to SOLR and HDFS.

I am not inclined to help you down a road you do not want to travel. There b=
e dragons!

Andy

> On 20 Aug 2020, at 07:25, Prashant Jyoti <[email protected]> wrote:
>=20
> =EF=BB=BFHi Joe,
> These are the errors I am running into:
>=20
> org.apache.solr.common.SolrException: Error CREATEing SolrCore
> 'newcollsolr2_shard1_replica_n1': Unable to create core
> [newcollsolr2_shard1_replica_n1] Caused by: Illegal char <:> at index 4:
> hdfs://
> hn1-pjhado.tvbhpqtgh3judk1e5ihrx2k21d.tx.internal.cloudapp.net:8020/user/s=
olr-data/newcollsolr2/core_node3/data\
> at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1256)
> at
> org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdmin=
Operation.java:93)
> at
> org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperatio=
n.java:362)
> at
> org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHand=
ler.java:397)
> at
> org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdmin=
Handler.java:181)
> at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBas=
e.java:211)
> at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:842)=

> at
> org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:=
808)
> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:559)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.jav=
a:420)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.jav=
a:352)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandl=
er.java:1596)
> at
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:545)=

> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:1=
43)
> at
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:590=
)
> at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java=
:127)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.ja=
va:235)
> at
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.ja=
va:1607)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.ja=
va:233)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.ja=
va:1297)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.jav=
a:188)
> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:48=
5)
> at
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.jav=
a:1577)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.jav=
a:186)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.jav=
a:1212)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:1=
41)
> at
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHa=
ndlerCollection.java:221)
> at
> org.eclipse.jetty.server.handler.InetAccessHandler.handle(InetAccessHandle=
r.java:177)
> at
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollectio=
n.java:146)
> at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java=
:127)
> at
> org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.jav=
a:322)
> at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java=
:127)
> at org.eclipse.jetty.server.Server.handle(Server.java:500)
> at
> org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383)=

> at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:547)
> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375)
> at
> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:270=
)
> at
> org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractCon=
nection.java:311)
> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
> at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKi=
ll.java:336)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYou=
Kill.java:313)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYo=
uKill.java:171)
> at
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.j=
ava:129)
> at
> org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(Re=
servedThreadExecutor.java:388)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.jav=
a:806)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool=
.java:938)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.apache.solr.common.SolrException: Unable to create core
> [newcollsolr2_shard1_replica_n1]
> at
> org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java=
:1327)
> at org.apache.solr.core.CoreContainer.create(CoreContainer.java:1217)
> ... 47 more
> Caused by: org.apache.solr.common.SolrException: Illegal char <:> at index=

> 4: hdfs://
> hn1-pjhado.tvbhpqtgh3judk1e5ihrx2k21d.tx.internal.cloudapp.net:8020/user/s=
olr-data/newcollsolr2/core_node3/data\
> at org.apache.solr.core.SolrCore.<init>(SolrCore.java:1072)
> at org.apache.solr.core.SolrCore.<init>(SolrCore.java:901)
> at
> org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java=
:1306)
> ... 48 more
> Caused by: java.nio.file.InvalidPathException: Illegal char <:> at index 4=
:
> hdfs://
> hn1-pjhado.tvbhpqtgh3judk1e5ihrx2k21d.tx.internal.cloudapp.net:8020/user/s=
olr-data/newcollsolr2/core_node3/data\
> at sun.nio.fs.WindowsPathParser.normalize(WindowsPathParser.java:182)
> at sun.nio.fs.WindowsPathParser.parse(WindowsPathParser.java:153)
> at sun.nio.fs.WindowsPathParser.parse(WindowsPathParser.java:77)
> at sun.nio.fs.WindowsPath.parse(WindowsPath.java:94)
> at sun.nio.fs.WindowsFileSystem.getPath(WindowsFileSystem.java:255)
> at sun.nio.fs.AbstractPath.resolve(AbstractPath.java:53)
> at org.apache.solr.core.SolrCore.initUpdateLogDir(SolrCore.java:1380)
> at org.apache.solr.core.SolrCore.<init>(SolrCore.java:958)
> ... 50 more
>=20
> Attaching it as a file too. Thanks!
>=20
>> On Wed, Aug 19, 2020 at 9:37 PM Joe Obernberger <
>> [email protected]> wrote:
>>=20
>> Your exception didn't come across - can you paste it in?
>>=20
>> -Joe
>>=20
>>> On 8/19/2020 10:50 AM, Prashant Jyoti wrote:
>>> You're right Andrew. Even I read about that. But there's a use case for
>>> which we want to configure the said case.
>>>=20
>>> Are you also aware of what feature we are moving towards instead of HDFS=
?
>>> Will you be able to help me with the error that I'm running into?
>>>=20
>>> Thanks in advance!
>>>=20
>>>=20
>>> On Wed, 19 Aug, 2020, 5:24 pm Andrew MacKay, <[email protected]>=

>>> wrote:
>>>=20
>>>> I believe HDFS support is being deprecated in Solr.  Not sure you want
>> to
>>>> continue configuration if support will disappear.
>>>>=20
>>>> On Wed, Aug 19, 2020 at 7:52 AM Prashant Jyoti <[email protected]>
>>>> wrote:
>>>>=20
>>>>> Hi all,
>>>>> Hope you are healthy and safe.
>>>>>=20
>>>>> Need some help with HDFS configuration.
>>>>>=20
>>>>> Could anybody of you share an example of the configuration with which
>> you
>>>>> are running Solr with HDFS in any of your production/dev environments?=

>>>>> I am interested in the parts of SolrConfig.xml / Solr.in.cmd/sh which
>> you
>>>>> may have modified. Obviously with the security parts obfuscated.
>>>>>=20
>>>>> I am stuck at an error and unable to move ahead. Attaching the
>> exception
>>>>> log if anyone is interested to look at the error.
>>>>>=20
>>>>> Thanks!
>>>>>=20
>>>>> --
>>>>> Regards,
>>>>> Prashant.
>>>>>=20
>>>> --
>>>> CONFIDENTIALITY NOTICE: The information contained in this email is
>>>> privileged and confidential and intended only for the use of the
>>>> individual
>>>> or entity to whom it is addressed.   If you receive this message in
>> error,
>>>> please notify the sender immediately at 613-729-1100 and destroy the
>>>> original message and all copies. Thank you.
>>>>=20
>>>=20
>>=20
>=20
>=20
> --=20
> Regards,
> Prashant.