Basic problem:
SEVERE: Could not start SOLR. Check solr/home property
java.lang.RuntimeException: java.io.FileNotFoundException: /solr_data/data/index/segments_l2b (No such file or directory)
at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1051)
at org.apache.solr.core.SolrCore.(SolrCore.java:565)
at org.apache.solr.core.CoreContainer$Initializer.initialize(CoreContainer.java:130)
at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:83)
at org.apache.catalina.core.ApplicationFilterConfig.getFilter(ApplicationFilterConfig.java:275)
at org.apache.catalina.core.ApplicationFilterConfig.setFilterDef(ApplicationFilterConfig.java:397)
at org.apache.catalina.core.ApplicationFilterConfig.(ApplicationFilterConfig.java:108)
at org.apache.catalina.core.StandardContext.filterStart(StandardContext.java:3800)
at org.apache.catalina.core.StandardContext.start(StandardContext.java:4450)
at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:791)
at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:771)
at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:526)
at org.apache.catalina.startup.HostConfig.deployDescriptor(HostConfig.java:630)
at org.apache.catalina.startup.HostConfig.deployDescriptors(HostConfig.java:556)
at org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:491)
at org.apache.catalina.startup.HostConfig.start(HostConfig.java:1206)
at org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:314)
at org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:119)
at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1053)
at org.apache.catalina.core.StandardHost.start(StandardHost.java:722)
at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1045)
at org.apache.catalina.core.StandardEngine.start(StandardEngine.java:443)
at org.apache.catalina.core.StandardService.start(StandardService.java:516)
at org.apache.catalina.core.StandardServer.start(StandardServer.java:710)
at org.apache.catalina.startup.Catalina.start(Catalina.java:583)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:288)
at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:413)
Caused by: java.io.FileNotFoundException: /solr_data/data/index/segments_l2b (No such file or directory)
at java.io.RandomAccessFile.open(Native Method)
at java.io.RandomAccessFile.(RandomAccessFile.java:212)
at org.apache.lucene.store.SimpleFSDirectory$SimpleFSIndexInput$Descriptor.(SimpleFSDirectory.java:77)
at org.apache.lucene.store.SimpleFSDirectory$SimpleFSIndexInput.(SimpleFSDirectory.java:107)
at org.apache.lucene.store.NIOFSDirectory$NIOFSIndexInput.(NIOFSDirectory.java:93)
at org.apache.lucene.store.NIOFSDirectory.openInput(NIOFSDirectory.java:69)
at org.apache.lucene.store.FSDirectory.openInput(FSDirectory.java:671)
at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:236)
at org.apache.lucene.index.DirectoryReader$1.doBody(DirectoryReader.java:71)
at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:704)
at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:68)
at org.apache.lucene.index.IndexReader.open(IndexReader.java:468)
at org.apache.lucene.index.IndexReader.open(IndexReader.java:309)
at org.apache.solr.core.StandardIndexReaderFactory.newReader(StandardIndexReaderFactory.java:37)
at org.apache.solr.core.SolrCore.getSearcher(SolrCore.java:1040)
... 30 more
Initial diagnosis:
Initial diagnosis showed that the segments_N file did not exist. The CheckIndex tool would not run, it reported the same problem as above. I had a bunch of index data files with lots of data in them:
-rw-r--r-- 1 root root 8028603002 Jul 3 20:27 _6l0.fdt
-rw-r--r-- 1 root root 14156172 Jul 3 20:27 _6l0.fdx
-rw-r--r-- 1 root root 328 Jul 3 20:21 _6l0.fnm
-rw-r--r-- 1 root root 869412416 Jul 3 20:33 _6l0.frq
-rw-r--r-- 1 root root 7078088 Jul 3 20:33 _6l0.nrm
-rw-r--r-- 1 root root 1749950441 Jul 3 20:33 _6l0.prx
-rw-r--r-- 1 root root 2428589 Jul 3 20:33 _6l0.tii
-rw-r--r-- 1 root root 194919244 Jul 3 20:33 _6l0.tis
-rw-r--r-- 1 root root 6867338 Jul 3 20:34 _6l0.tvd
-rw-r--r-- 1 root root 1516183431 Jul 3 20:34 _6l0.tvf
-rw-r--r-- 1 root root 28312340 Jul 3 20:34 _6l0.tvx
-rw-r--r-- 1 root root 3943 Jul 7 17:00 _6l0_14u.del
-rw-r--r-- 1 root root 3588947105 Jul 4 10:16 _a64.fdt
-rw-r--r-- 1 root root 5950556 Jul 4 10:16 _a64.fdx
-rw-r--r-- 1 root root 328 Jul 4 10:13 _a64.fnm
-rw-r--r-- 1 root root 388042305 Jul 4 10:19 _a64.frq
-rw-r--r-- 1 root root 2975280 Jul 4 10:19 _a64.nrm
-rw-r--r-- 1 root root 786393928 Jul 4 10:19 _a64.prx
-rw-r--r-- 1 root root 1332607 Jul 4 10:19 _a64.tii
-rw-r--r-- 1 root root 106349618 Jul 4 10:19 _a64.tis
-rw-r--r-- 1 root root 2898023 Jul 4 10:21 _a64.tvd
-rw-r--r-- 1 root root 660740410 Jul 4 10:21 _a64.tvf
-rw-r--r-- 1 root root 11901108 Jul 4 10:21 _a64.tvx
-rw-r--r-- 1 root root 427 Jul 7 16:55 _a64_3x.del
-rw-r--r-- 1 root root 4063439055 Jul 5 08:35 _fyc.fdt
-rw-r--r-- 1 root root 6684556 Jul 5 08:35 _fyc.fdx
-rw-r--r-- 1 root root 328 Jul 5 08:30 _fyc.fnm
-rw-r--r-- 1 root root 442166844 Jul 5 08:39 _fyc.frq
-rw-r--r-- 1 root root 3342280 Jul 5 08:39 _fyc.nrm
-rw-r--r-- 1 root root 873120822 Jul 5 08:39 _fyc.prx
-rw-r--r-- 1 root root 1535756 Jul 5 08:39 _fyc.tii
-rw-r--r-- 1 root root 123009510 Jul 5 08:39 _fyc.tis
-rw-r--r-- 1 root root 3171898 Jul 5 08:40 _fyc.tvd
-rw-r--r-- 1 root root 760721721 Jul 5 08:40 _fyc.tvf
-rw-r--r-- 1 root root 13369108 Jul 5 08:40 _fyc.tvx
-rw-r--r-- 1 root root 340 Jul 7 16:55 _fyc_35.del
-rw-r--r-- 1 root root 3774135320 Jul 6 08:30 _lwv.fdt
-rw-r--r-- 1 root root 6003948 Jul 6 08:30 _lwv.fdx
-rw-r--r-- 1 root root 328 Jul 6 08:26 _lwv.fnm
-rw-r--r-- 1 root root 418485969 Jul 6 08:34 _lwv.frq
-rw-r--r-- 1 root root 3001976 Jul 6 08:34 _lwv.nrm
-rw-r--r-- 1 root root 843268283 Jul 6 08:34 _lwv.prx
-rw-r--r-- 1 root root 1385423 Jul 6 08:34 _lwv.tii
-rw-r--r-- 1 root root 111141259 Jul 6 08:34 _lwv.tis
-rw-r--r-- 1 root root 2820840 Jul 6 08:36 _lwv.tvd
-rw-r--r-- 1 root root 717490764 Jul 6 08:36 _lwv.tvf
-rw-r--r-- 1 root root 12007892 Jul 6 08:36 _lwv.tvx
-rw-r--r-- 1 root root 38 Jul 7 16:54 _lwv_8.del
-rw-r--r-- 1 root root 404238685 Jul 6 11:23 _mlu.fdt
-rw-r--r-- 1 root root 208748 Jul 6 11:23 _mlu.fdx
-rw-r--r-- 1 root root 227 Jul 6 11:23 _mlu.fnm
-rw-r--r-- 1 root root 46234520 Jul 6 11:24 _mlu.frq
-rw-r--r-- 1 root root 78283 Jul 6 11:24 _mlu.nrm
-rw-r--r-- 1 root root 106629909 Jul 6 11:24 _mlu.prx
-rw-r--r-- 1 root root 196486 Jul 6 11:24 _mlu.tii
-rw-r--r-- 1 root root 15606446 Jul 6 11:24 _mlu.tis
-rw-r--r-- 1 root root 51755 Jul 6 11:24 _mlu.tvd
-rw-r--r-- 1 root root 78839439 Jul 6 11:24 _mlu.tvf
-rw-r--r-- 1 root root 417492 Jul 6 11:24 _mlu.tvx
-rw-r--r-- 1 root root 15 Jul 6 12:14 _mlu_1.del
-rw-r--r-- 1 root root 309050678 Jul 7 16:13 _n6y.fdt
-rw-r--r-- 1 root root 165844 Jul 7 16:13 _n6y.fdx
-rw-r--r-- 1 root root 227 Jul 7 16:13 _n6y.fnm
-rw-r--r-- 1 root root 36044993 Jul 7 16:14 _n6y.frq
-rw-r--r-- 1 root root 62194 Jul 7 16:14 _n6y.nrm
-rw-r--r-- 1 root root 81168504 Jul 7 16:14 _n6y.prx
-rw-r--r-- 1 root root 161925 Jul 7 16:14 _n6y.tii
-rw-r--r-- 1 root root 12783522 Jul 7 16:14 _n6y.tis
-rw-r--r-- 1 root root 41096 Jul 7 16:14 _n6y.tvd
-rw-r--r-- 1 root root 61007940 Jul 7 16:14 _n6y.tvf
-rw-r--r-- 1 root root 331684 Jul 7 16:14 _n6y.tvx
-rw-r--r-- 1 root root 190234359 Jul 7 16:45 _nbf.fdt
-rw-r--r-- 1 root root 272564 Jul 7 16:45 _nbf.fdx
-rw-r--r-- 1 root root 328 Jul 7 16:45 _nbf.fnm
-rw-r--r-- 1 root root 20098506 Jul 7 16:45 _nbf.frq
-rw-r--r-- 1 root root 136284 Jul 7 16:45 _nbf.nrm
-rw-r--r-- 1 root root 43187014 Jul 7 16:45 _nbf.prx
-rw-r--r-- 1 root root 103893 Jul 7 16:45 _nbf.tii
-rw-r--r-- 1 root root 8123849 Jul 7 16:45 _nbf.tis
-rw-r--r-- 1 root root 126312 Jul 7 16:45 _nbf.tvd
-rw-r--r-- 1 root root 35035417 Jul 7 16:45 _nbf.tvf
-rw-r--r-- 1 root root 545124 Jul 7 16:45 _nbf.tvx
-rw-r--r-- 1 root root 82107318 Jul 7 16:55 _nd8.fdt
-rw-r--r-- 1 root root 136068 Jul 7 16:55 _nd8.fdx
-rw-r--r-- 1 root root 328 Jul 7 16:55 _nd8.fnm
-rw-r--r-- 1 root root 8642891 Jul 7 16:55 _nd8.frq
-rw-r--r-- 1 root root 68036 Jul 7 16:55 _nd8.nrm
-rw-r--r-- 1 root root 17964715 Jul 7 16:55 _nd8.prx
-rw-r--r-- 1 root root 60156 Jul 7 16:55 _nd8.tii
-rw-r--r-- 1 root root 4671427 Jul 7 16:55 _nd8.tis
-rw-r--r-- 1 root root 69422 Jul 7 16:56 _nd8.tvd
-rw-r--r-- 1 root root 15417989 Jul 7 16:56 _nd8.tvf
-rw-r--r-- 1 root root 272132 Jul 7 16:56 _nd8.tvx
-rw-r--r-- 1 root root 46219469 Jul 7 16:58 _ndo.fdt
-rw-r--r-- 1 root root 62964 Jul 7 16:58 _ndo.fdx
-rw-r--r-- 1 root root 328 Jul 7 16:58 _ndo.fnm
-rw-r--r-- 1 root root 4357258 Jul 7 16:58 _ndo.frq
-rw-r--r-- 1 root root 31484 Jul 7 16:58 _ndo.nrm
-rw-r--r-- 1 root root 10196509 Jul 7 16:58 _ndo.prx
-rw-r--r-- 1 root root 33580 Jul 7 16:58 _ndo.tii
-rw-r--r-- 1 root root 2566267 Jul 7 16:58 _ndo.tis
-rw-r--r-- 1 root root 31172 Jul 7 16:58 _ndo.tvd
-rw-r--r-- 1 root root 7756092 Jul 7 16:58 _ndo.tvf
-rw-r--r-- 1 root root 125924 Jul 7 16:58 _ndo.tvx
-rw-r--r-- 1 root root 31234528 Jul 7 17:00 _ndz.fdt
-rw-r--r-- 1 root root 38196 Jul 7 17:00 _ndz.fdx
-rw-r--r-- 1 root root 328 Jul 7 17:00 _ndz.fnm
-rw-r--r-- 1 root root 2689685 Jul 7 17:00 _ndz.frq
-rw-r--r-- 1 root root 19100 Jul 7 17:00 _ndz.nrm
-rw-r--r-- 1 root root 6729134 Jul 7 17:00 _ndz.prx
-rw-r--r-- 1 root root 20814 Jul 7 17:00 _ndz.tii
-rw-r--r-- 1 root root 1572257 Jul 7 17:00 _ndz.tis
-rw-r--r-- 1 root root 18936 Jul 7 17:00 _ndz.tvd
-rw-r--r-- 1 root root 4777281 Jul 7 17:00 _ndz.tvf
-rw-r--r-- 1 root root 76388 Jul 7 17:00 _ndz.tvx
-rw-r--r-- 1 root root 15 Jul 7 17:00 _ndz_1.del
-rw-r--r-- 1 root root 3143760 Jul 7 17:00 _ne0.fdt
-rw-r--r-- 1 root root 1004 Jul 7 17:00 _ne0.fdx
-rw-r--r-- 1 root root 328 Jul 7 17:00 _ne0.fnm
-rw-r--r-- 1 root root 242609 Jul 7 17:00 _ne0.frq
-rw-r--r-- 1 root root 504 Jul 7 17:00 _ne0.nrm
-rw-r--r-- 1 root root 820393 Jul 7 17:00 _ne0.prx
-rw-r--r-- 1 root root 4440 Jul 7 17:00 _ne0.tii
-rw-r--r-- 1 root root 333653 Jul 7 17:00 _ne0.tis
-rw-r--r-- 1 root root 351 Jul 7 17:00 _ne0.tvd
-rw-r--r-- 1 root root 473525 Jul 7 17:00 _ne0.tvf
-rw-r--r-- 1 root root 2004 Jul 7 17:00 _ne0.tvx
-rw-r--r-- 1 root root 24 Jul 7 17:00 _ne0_2.del
-rw-r--r-- 1 root root 2800555 Jul 7 17:00 _ne1.fdt
-rw-r--r-- 1 root root 1148 Jul 7 17:00 _ne1.fdx
-rw-r--r-- 1 root root 328 Jul 7 17:00 _ne1.fnm
-rw-r--r-- 1 root root 222294 Jul 7 17:00 _ne1.frq
-rw-r--r-- 1 root root 576 Jul 7 17:00 _ne1.nrm
-rw-r--r-- 1 root root 749837 Jul 7 17:00 _ne1.prx
-rw-r--r-- 1 root root 4109 Jul 7 17:00 _ne1.tii
-rw-r--r-- 1 root root 294783 Jul 7 17:00 _ne1.tis
-rw-r--r-- 1 root root 448 Jul 7 17:00 _ne1.tvd
-rw-r--r-- 1 root root 421801 Jul 7 17:00 _ne1.tvf
-rw-r--r-- 1 root root 2292 Jul 7 17:00 _ne1.tvx
-rw-r--r-- 1 root root 26 Jul 7 17:00 _ne1_1.del
Steps taken:
- Copied over valid segments.gen and segments_N files from another server with a non-corrupt index
- Ran the Lucene CheckIndex tool to "fix" the segments files (in /webapp/WEB-INF/lib):
java -cp lucene-core-2.9-dev.jar org.apache.lucene.index.CheckIndex /solr_data/data/index/
- Now I had a bunch of data files and empty segments files that did not point to my data files. So I started the solr server and ran
/solr/update?optimize=true
in the hopes that this would re-create the segments files and keep the data files intact
Results:
Optimize removed the data files so I was left with only segments.gen and segments_bao. This whole step failed and now I'll have to re-create the index (which took 5 days last time). Bummer.
0 comments:
Post a Comment