resolve as Fixed
|Assigned To||Salendra Singh||»||rwiseman|
Today's MiSeq run (#122 or #604 depending on which numbering system you use) still hasn't completed it's import into LabKey ~12hrs after the run finished so that we can get to the binned FASTQ files. This looks like the same behavior that we experienced with our last run - all of the metrics data is there when you click on the "file browser" tab under "view data" for MiSeq 122 but there's nothing under the "sample data" tab for reads associated with each of the 188 different barcodes/indexs/MIDs. The FASTQ files were binned on the MiSeq in the appropriate output folder on the MiSeq itself around 10AM this morning so the data is processed. Under the "Data Pipeline" tab it says "importing reads" was created at "2013-09-01 12:59" but it must be stuck again since it shouldn't take >9hrs for the FASTQ files to transfer. Do we need to restart LabKey again?
Diagnosis : All scripts working fine. Server processed data very slowly( Usually its 12500 reads/sec But specifically for this run and Manual import of Run No 97 it went down 210 reads/ sec( 12500 reads in a minute) . It may be because of excessive load on the server .
For Further Consideration this is the timeline
Data Transfer from MiSeq to Bonobo
[Sun Sep 1 12:30:01 2013] Copying MiSeq run directory, 130830_M01472_0069_000000000-A5R6E.
[Sun Sep 1 12:52:14 2013] Copying complete.
[Sun Sep 1 13:00:01 2013] Started Data Processing
Data Processing at Bonobo
Sun Sep 1 13:00:02 2013] Making run directory miseq122 and subdirectories
[Sun Sep 1 13:00:02 2013] Copying files from /media/RAID6-ext4/miseq_data/MiSeqOutput/130830_M01472_0069_000000000-A5R6E/Data/Intensities/BaseCalls to /media/RAID6-ext4/miseq_data/uploaded_runs/miseq122/BaseCalls
[Sun Sep 1 13:00:26 2013] Copying files from /media/RAID6-ext4/miseq_data/MiSeqOutput/130830_M01472_0069_000000000-A5R6E to /media/RAID6-ext4/miseq_data/uploaded_runs/miseq122
[Sun Sep 1 13:00:26 2013] Copying files from /media/RAID6-ext4/miseq_data/MiSeqOutput/130830_M01472_0069_000000000-A5R6E/InterOp to /media/RAID6-ext4/miseq_data/uploaded_runs/miseq122/InterOp
[Sun Sep 1 13:00:26 2013] Parsing /media/RAID6-ext4/miseq_data/uploaded_runs/miseq122/SampleSheet.csv header and uploading data into LabKey
[Sun Sep 1 13:00:26 2013] Starting import of Illumina Reads in LabKey
[Sun Sep 1 13:00:27 2013] Archiving and compressing raw run data directory
[Sun Sep 1 13:08:18 2013] Run processing complete
Data Processing on LabKey
Data Processing Started : 01 Sep 2013 12:59:35,055
Data Processsing Ended : 03 Sep 2013 09:14:03,990