PassMark Logo
Home » Forum

Announcement

Collapse
No announcement yet.

application crashed when incremental indexing started

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • application crashed when incremental indexing started

    I have four incremental indexing running at the same time, and two of them just crashed when a new round of incremental indexing started. I am using v6.0 1016 and the crashing windows error messages are as follows:

    Problem signature:
    Problem Event Name: APPCRASH
    Application Name: ZoomIndexer64.exe
    Application Version: 6.0.0.0
    Application Timestamp: 4a41c732
    Fault Module Name: ZoomIndexer64.exe
    Fault Module Version: 6.0.0.0
    Fault Module Timestamp: 4a41c732
    Exception Code: c0000005
    Exception Offset: 00000000000e683f
    OS Version: 6.0.6002.2.2.0.1296.17
    Locale ID: 2052
    Additional Information 1: 13a6
    Additional Information 2: a24a7b585eccdf0404c6826d940d59da
    Additional Information 3: f717
    Additional Information 4: d4263b8fc2689103ad7c9ec9ab04d364

    Read our privacy statement:
    http://go.microsoft.com/fwlink/?linkid=50163&clcid=0x0409


    Problem signature:
    Problem Event Name: APPCRASH
    Application Name: ZoomIndexer64.exe
    Application Version: 6.0.0.0
    Application Timestamp: 4a41c732
    Fault Module Name: ZoomIndexer64.exe
    Fault Module Version: 6.0.0.0
    Fault Module Timestamp: 4a41c732
    Exception Code: c0000005
    Exception Offset: 00000000000e683f
    OS Version: 6.0.6002.2.2.0.1296.17
    Locale ID: 2052
    Additional Information 1: 13a6
    Additional Information 2: a24a7b585eccdf0404c6826d940d59da
    Additional Information 3: f717
    Additional Information 4: d4263b8fc2689103ad7c9ec9ab04d364

    Read our privacy statement:
    http://go.microsoft.com/fwlink/?linkid=50163&clcid=0x0409

  • #2
    Can you contact us via e-mail with more information.

    I presume your simultaneous incremental indexes are writing to different output folders.

    What are the sizes involved (the number of pages indexed, the number of pages being added, the number of unique words, the total file size of the existing ZDAT files you are incrementally adding to).

    How much memory is available on the machine and how much is being utilized? (this is reported on the "Status" tab of the Indexer). If these are very big indexes, it is possible that your system memory is being exhausted, especially with multiple indexers running at the same time.

    Ideally, if possible, it would be most helpful to have the actual index files you are incrementally adding to, your ZCFG configuration file, and an accurate description of the incremental operation you are doing to trigger this crash (e.g. was it an "incremental add start points", or "incremental update", and what URLs you were adding, etc.). In other words, enough details for us to try and reproduce the behaviour here.

    Also, have you tried doing the same operation but not running multiple instances of the indexer at the same time, and did the crash occur?
    --Ray
    Wrensoft Web Software
    Sydney, Australia
    Zoom Search Engine

    Comment


    • #3
      I just tried running the crashed incremental indexing again with no other indexing operations, and they crashed again. I also tried with different urls and got same results.

      I was running from command line using "incremental adding pages" option for all the four incremental operations. I am not running out of memory and the crash happened after each successfully finishing 6 rounds of incremental indexing. The two operations that were running without problem were using the same ZCFG file with the crashed ones. The unique words were around 2,000,000 when the two indexing crashed and the pagetext files were 2GB.

      Comment


      • #4
        Ideally, if possible, it would be most helpful to have the actual index files you are incrementally adding to & your ZCFG configuration file. Can you zip them up & make them available via download.

        Even though you are using release v6.0.1016 of the software now, did you create the initial set of index files with an earlier release of the software?

        Comment


        • #5
          I cannot provide the files you requested. I have tested the indexed files and they are working fine. So I guess the zoomindexer64.exe file might be broken due to the crash. I just reinstalled the program and am now waiting to see what will happen. I found that the loading time for each new round of incremental indexing is very long, usually around 2 to 3 hours for 100,000 urls. Maybe four simultaneous loading of urls caused the application to crash? The initial set of index files were also created by v6.0 1016.

          Comment


          • #6
            A crash will not corrupt or break the executable.

            If you turn on debug mode logging ("Configure"->"Index Log"->"Debug mode" and make sure to SAVE the configuration change to the file you're running from the command line), then you should be able to provide us with log files to indicate when the crash occurred during the indexing process.

            If you can't give us the files (I presume due to data confidentiality and not because you don't know how -- but since you don't say, I'm guessing again), at least provide as much information as possible. There were many other things I mentioned before that were not answered: what were the number of files indexed (per index), the total size of all the ZDAT files (per index), the amount of memory you have installed on your machine. Please look at my previous post.

            Also tell us the version of Windows you are using. The hardware you are using.

            And I also suggested you try to see if it happens when you are not running multiple indexers simultaneously.

            If you can't give us the files, we can't reproduce it here. So you'd need to give us as much information as possible for us to look into it.
            --Ray
            Wrensoft Web Software
            Sydney, Australia
            Zoom Search Engine

            Comment


            • #7
              It's also worth pointing out that running several instances simultaneously, when dealing with such large data sets on the same computer, you are very likely to be slowing down each instance as it is bounded by the limitations of the hard drive speed, the ram, and the CPU available. I would suspect the several hours of loading should well improve if you're not running multiple instances at the same time.
              --Ray
              Wrensoft Web Software
              Sydney, Australia
              Zoom Search Engine

              Comment


              • #8
                The OS is windows web server 2008, and RAM is 8GB, Cpu is Intel Core 2 Duo E6600. The total number of files in each index was around 500,000 and total index file size was 2.94GB when the program crashed. Each index used about 1.4GB of memory. I tried running single index, and the crashed ones continued to crash and the working ones remained working. And reinstalling the program did not make a difference. I also tried version 1017, it just quickly opened and closed the program without doing anything and continued to the next round.

                Comment


                • #9
                  You can't use build 1017 to incrementally add to index files created with build 1016. An error message would have been reported in the log file, which inform you about this.

                  To try to reproduce this with the latest build, you would need to do a full re-index using the new build, and then incrementally add to the resultant files.

                  It is possible that the new build might have fixed this with the bug fix for the pipe characters, so it is worth checking.

                  If you can reproduce this with the latest build, enable the debug logging as mentioned above and send this to us.
                  --Ray
                  Wrensoft Web Software
                  Sydney, Australia
                  Zoom Search Engine

                  Comment

                  Working...
                  X