Unexpected error during the aligment: ERROR: ('wtf4!:pointer: %i', 20)

[fanavarro]

I am trying to use CRISPResso2, but most of the times I am getting an unexpected error. For example, for the command:

CRISPResso --debug --exclude_bp_from_right 0 --exclude_bp_from_left 0 --fastq_r1 fastq_uge_crispresso/aa5e8c0dc2f860f3bbca5f14ba36b127b19aa3d0_SQK-NBD114-24_barcode21.fastq --output_folder output --file_prefix barcode21 --zip_output --amplicon_seq actcagagacacctgtgtgg

I am getting the following output:

INFO  @ Sat, 03 May 2025 11:38:39 (0.0% done):
	 
                               ~~~CRISPResso 2~~~                               
        -Analysis of genome editing outcomes from deep sequencing data-         
                                                                                
                                        _                                       
                                       '  )                                     
                                       .-'                                      
                                      (____                                     
                                   C)|     \                                    
                                     \     /                                    
                                      \___/                                     

                           [CRISPResso version 2.3.2]                           
[Note that as of version 2.3.0 FLASh and Trimmomatic have been replaced by fastp for read merging and trimming. Accordingly, the --flash_command and --trimmomatic_command parameters have been replaced with --fastp_command. Also, --trimmomatic_options_string has been replaced with --fastp_options_string.

Also in version 2.3.2, when running CRISPRessoPooled in mixed-mode (amplicon file and genome are provided) the default behavior will be as if the --demultiplex_only_at_amplicons parameter is provided. This change means that reads and amplicons do not need to align to the exact locations.]
       [For support contact k.clement@utah.edu or support@edilytics.com]        
 

WARNING @ Sat, 03 May 2025 11:38:39 (0.0% done):
	 Folder /pool_sas/longseqservice/pruebas_fran/crispresso/output/CRISPResso_on_aa5e8c0dc2f860f3bbca5f14ba36b127b19aa3d0_SQK-NBD114-24_barcode21 already exists. 

DEBUG @ Sat, 03 May 2025 11:38:39 (0.5% done):
	 CRISPRessoPro not installed 

INFO  @ Sat, 03 May 2025 11:38:39 (2.0% done):
	 Computing quantification windows 

DEBUG @ Sat, 03 May 2025 11:38:39 (7.0% done):
	 Added 0 guides with flexible matching
	Original flexiguides: ['None']
	Found guides: []
	Mismatch locations: [] 

INFO  @ Sat, 03 May 2025 11:38:40 (7.0% done):
	 Aligning sequences... 

INFO  @ Sat, 03 May 2025 11:38:40 (7.0% done):
	 Finished reading fastq file; 37173 unique reads found of 37604 total reads found  

INFO  @ Sat, 03 May 2025 11:38:40 (7.0% done):
	 Processing Reads; 0 Completed out of 37173 Unique Reads 

CRITICAL @ Sat, 03 May 2025 11:38:40 (7.0% done):
	 Traceback (most recent call last):
  File "/home/longseqservice/mambaforge/envs/crispresso2/lib/python3.12/site-packages/CRISPResso2/CRISPRessoCORE.py", line 2626, in main
    aln_stats = process_fastq(processed_output_filename, variantCache, ref_names, refs, args, files_to_remove, OUTPUT_DIRECTORY)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/longseqservice/mambaforge/envs/crispresso2/lib/python3.12/site-packages/CRISPResso2/CRISPRessoCORE.py", line 728, in process_fastq
    variant = get_new_variant_object(args, fastq_seq, refs, ref_names, aln_matrix, pe_scaffold_dna_info)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/longseqservice/mambaforge/envs/crispresso2/lib/python3.12/site-packages/CRISPResso2/CRISPRessoCORE.py", line 310, in get_new_variant_object
    fws1, fws2, fwscore=CRISPResso2Align.global_align(fastq_seq, refs[ref_name]['sequence'], matrix=aln_matrix, gap_incentive=refs[ref_name]['gap_incentive'], gap_open=args.needleman_wunsch_gap_open, gap_extend=args.needleman_wunsch_gap_extend,)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "CRISPResso2/CRISPResso2Align.pyx", line 418, in CRISPResso2.CRISPResso2Align.global_align
Exception: ('wtf4!:pointer: %i', 20)
 

CRITICAL @ Sat, 03 May 2025 11:38:40 (7.0% done):
	 Unexpected error, please check your input.

ERROR: ('wtf4!:pointer: %i', 20) 

i: 20 j: 0
currMatrix:96
seqj: b'' seqi: b'ACTCAGAGACACCTGTGTGG'
Traceback (most recent call last):
  File "/home/longseqservice/mambaforge/envs/crispresso2/lib/python3.12/site-packages/CRISPResso2/CRISPRessoCORE.py", line 2626, in main
    aln_stats = process_fastq(processed_output_filename, variantCache, ref_names, refs, args, files_to_remove, OUTPUT_DIRECTORY)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/longseqservice/mambaforge/envs/crispresso2/lib/python3.12/site-packages/CRISPResso2/CRISPRessoCORE.py", line 728, in process_fastq
    variant = get_new_variant_object(args, fastq_seq, refs, ref_names, aln_matrix, pe_scaffold_dna_info)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/longseqservice/mambaforge/envs/crispresso2/lib/python3.12/site-packages/CRISPResso2/CRISPRessoCORE.py", line 310, in get_new_variant_object
    fws1, fws2, fwscore=CRISPResso2Align.global_align(fastq_seq, refs[ref_name]['sequence'], matrix=aln_matrix, gap_incentive=refs[ref_name]['gap_incentive'], gap_open=args.needleman_wunsch_gap_open, gap_extend=args.needleman_wunsch_gap_extend,)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "CRISPResso2/CRISPResso2Align.pyx", line 418, in CRISPResso2.CRISPResso2Align.global_align
Exception: ('wtf4!:pointer: %i', 20)

Do you have any clue about what could be happening here?

Thanks beforehand.

[kclem]

Hi @fanavarro This looks like an error with the alignment. I'm guessing it has to do with the read length or perhaps some weird characters in the fastq file.

It would be helpful if we could get a copy of the fastq file for troubleshooting on our side.
It looks like it happens in the first few sequences in your file. Do you mind creating a short version of your input fastq file:

head -n 50 fastq_uge_crispresso/aa5e8c0dc2f860f3bbca5f14ba36b127b19aa3d0_SQK-NBD114-24_barcode21.fastq > test1.head.fq

Do you get the same error on this small file? If so, do you mind sharing the small file here?

[fanavarro]

Hi, thanks for your answer. I have taken the first 52 lines to get al the lines of the last read, but then I got this error:

INFO  @ Thu, 08 May 2025 11:54:06 (0.0% done):
	 
                               ~~~CRISPResso 2~~~                               
        -Analysis of genome editing outcomes from deep sequencing data-         
                                                                                
                                        _                                       
                                       '  )                                     
                                       .-'                                      
                                      (____                                     
                                   C)|     \                                    
                                     \     /                                    
                                      \___/                                     

                           [CRISPResso version 2.3.2]                           
[Note that as of version 2.3.0 FLASh and Trimmomatic have been replaced by fastp for read merging and trimming. Accordingly, the --flash_command and --trimmomatic_command parameters have been replaced with --fastp_command. Also, --trimmomatic_options_string has been replaced with --fastp_options_string.

Also in version 2.3.2, when running CRISPRessoPooled in mixed-mode (amplicon file and genome are provided) the default behavior will be as if the --demultiplex_only_at_amplicons parameter is provided. This change means that reads and amplicons do not need to align to the exact locations.]
       [For support contact k.clement@utah.edu or support@edilytics.com]        
 

INFO  @ Thu, 08 May 2025 11:54:06 (0.0% done):
	 Creating Folder /pool_sas/longseqservice/pruebas_fran/crispresso/output/CRISPResso_on_test1.head 

DEBUG @ Thu, 08 May 2025 11:54:06 (0.5% done):
	 CRISPRessoPro not installed 

INFO  @ Thu, 08 May 2025 11:54:06 (2.0% done):
	 Computing quantification windows 

DEBUG @ Thu, 08 May 2025 11:54:06 (7.0% done):
	 Added 0 guides with flexible matching
	Original flexiguides: ['None']
	Found guides: []
	Mismatch locations: [] 

INFO  @ Thu, 08 May 2025 11:54:06 (7.0% done):
	 Aligning sequences... 

INFO  @ Thu, 08 May 2025 11:54:06 (7.0% done):
	 Finished reading fastq file; 13 unique reads found of 13 total reads found  

INFO  @ Thu, 08 May 2025 11:54:06 (7.0% done):
	 Processing Reads; 0 Completed out of 13 Unique Reads 

INFO  @ Thu, 08 May 2025 11:54:06 (7.0% done):
	 Finished reads; N_TOT_READS: 13 N_COMPUTED_ALN: 0 N_CACHED_ALN: 0 N_COMPUTED_NOTALN: 13 N_CACHED_NOTALN: 0 

INFO  @ Thu, 08 May 2025 11:54:06 (20.0% done):
	 Done! 

INFO  @ Thu, 08 May 2025 11:54:06 (20.0% done):
	 Quantifying indels/substitutions... 

INFO  @ Thu, 08 May 2025 11:54:06 (30.0% done):
	 Done! 

CRITICAL @ Thu, 08 May 2025 11:54:06 (30.0% done):
	 Traceback (most recent call last):
  File "/home/longseqservice/mambaforge/envs/crispresso2/lib/python3.12/site-packages/CRISPResso2/CRISPRessoCORE.py", line 3186, in main
    raise CRISPRessoShared.NoReadsAlignedException('Error: No alignments were found')
CRISPResso2.CRISPRessoShared.NoReadsAlignedException: Error: No alignments were found
 

CRITICAL @ Thu, 08 May 2025 11:54:06 (30.0% done):
	 Alignment error, please check your input.

ERROR: Error: No alignments were found 

Traceback (most recent call last):
  File "/home/longseqservice/mambaforge/envs/crispresso2/lib/python3.12/site-packages/CRISPResso2/CRISPRessoCORE.py", line 3186, in main
    raise CRISPRessoShared.NoReadsAlignedException('Error: No alignments were found')
CRISPResso2.CRISPRessoShared.NoReadsAlignedException: Error: No alignments were found

Maybe there are not enough reads to perform the alignment with this small file?

[kclem]

I have seen this error before when the process runs out of memory. Does this problem still occur with the full file on a larger machine with more memory?

[fanavarro]

Hi @kclem I do not think it is a memory issue; our server has 500GB of RAM memory. In addition, I run this in my laptop (32GB RAM) while monitoring the memory usage, but it failed almost instantly without memory consumption.

The fastq file I am using is private, but I can share with you by email the file I am using if you want it only for testing CIRSPResso.

Other point of interest is that my colleague used this file before in the CRISPResso web application and she said it worked; however, she tried to run it again more recently and it failed. In particular, the part of the code that is failing is the alignment; the code checks several if, else if, else conditions, and if the code flow goes in the else statement, it throw the exception:

        else:
            print('i: ' + str(i) + ' j: ' + str(j))
            print('currMatrix:' + str(currMatrix))
            print('seqj: ' + str(seqj) + ' seqi: ' + str(seqi))
            raise Exception('wtf4!:pointer: %i', i)

You can find this code at the end of the function global_align in CRISPResso2Align.pyx

Tell me if you need me to do some test or something if you need.

Kind regards.

[kclem]

Sure, if you can create a small fastq file that reproduces the error and send it to k.clement@utah.edu I can look at it.

[zuoyichen]

Hello, I’ve encountered the same error — ('wtf4!:pointer: %i', 210) — when running CRISPResso2. Has anyone found a solution or workaround for this bug?

[fanavarro]

@zuoyichen I shared my fastq file with @kclem so he is able to reproduce the error, but we don't have any solution yet.

[zuoyichen]

@zuoyichen我与他人分享了我的 fastq 文件@kclem所以他能够重现该错误，但我们还没有任何解决方案。

Hi, I think this bug might have been caused by too many "N" bases in the reads. After I used cutadapt --max-n to filter out such reads, the processed FASTQ ran without any issues.

[fanavarro]

Hi, after reading the @zuoyichen comment I have inspected the reads I sent to @kclem and I saw that the fastq file contained weird reads (repeated identifiers and reads without sequence). After removing these strange reads, CRISPResso2 failed with "no aligment found" error:

	 
                               ~~~CRISPResso 2~~~                               
        -Analysis of genome editing outcomes from deep sequencing data-         
                                                                                
                                        _                                       
                                       '  )                                     
                                       .-'                                      
                                      (____                                     
                                   C)|     \                                    
                                     \     /                                    
                                      \___/                                     

                           [CRISPResso version 2.3.2]                           
[Note that as of version 2.3.0 FLASh and Trimmomatic have been replaced by fastp for read merging and trimming. Accordingly, the --flash_command and --trimmomatic_command parameters have been replaced with --fastp_command. Also, --trimmomatic_options_string has been replaced with --fastp_options_string.

Also in version 2.3.2, when running CRISPRessoPooled in mixed-mode (amplicon file and genome are provided) the default behavior will be as if the --demultiplex_only_at_amplicons parameter is provided. This change means that reads and amplicons do not need to align to the exact locations.]
       [For support contact k.clement@utah.edu or support@edilytics.com]        
 

WARNING @ Tue, 10 Jun 2025 09:43:50 (0.0% done):
	 Folder /home/fabad/Descargas/crispresso_test/output/CRISPResso_on_aa5e8c0dc2f860f3bbca5f14ba36b127b19aa3d0_SQK-NBD114-24_barcode21_clean already exists. 

DEBUG @ Tue, 10 Jun 2025 09:43:50 (0.5% done):
	 CRISPRessoPro not installed 

INFO  @ Tue, 10 Jun 2025 09:43:50 (2.0% done):
	 Computing quantification windows 

DEBUG @ Tue, 10 Jun 2025 09:43:50 (7.0% done):
	 Added 0 guides with flexible matching
	Original flexiguides: ['None']
	Found guides: []
	Mismatch locations: [] 

INFO  @ Tue, 10 Jun 2025 09:43:50 (7.0% done):
	 Aligning sequences... 

INFO  @ Tue, 10 Jun 2025 09:43:50 (7.0% done):
	 Finished reading fastq file; 30310 unique reads found of 30704 total reads found  

INFO  @ Tue, 10 Jun 2025 09:43:50 (7.0% done):
	 Processing Reads; 0 Completed out of 30310 Unique Reads 

INFO  @ Tue, 10 Jun 2025 09:43:52 (7.0% done):
	 Processing Reads; 10000 Completed out of 30310 Unique Reads 

INFO  @ Tue, 10 Jun 2025 09:43:54 (7.0% done):
	 Processing Reads; 20000 Completed out of 30310 Unique Reads 

INFO  @ Tue, 10 Jun 2025 09:43:56 (7.0% done):
	 Processing Reads; 30000 Completed out of 30310 Unique Reads 

INFO  @ Tue, 10 Jun 2025 09:43:56 (7.0% done):
	 Finished reads; N_TOT_READS: 30704 N_COMPUTED_ALN: 0 N_CACHED_ALN: 0 N_COMPUTED_NOTALN: 30310 N_CACHED_NOTALN: 394 

INFO  @ Tue, 10 Jun 2025 09:43:56 (20.0% done):
	 Done! 

INFO  @ Tue, 10 Jun 2025 09:43:56 (20.0% done):
	 Quantifying indels/substitutions... 

INFO  @ Tue, 10 Jun 2025 09:43:56 (30.0% done):
	 Done! 

Traceback (most recent call last):
  File "/home/fabad/mambaforge/envs/crispresso2/lib/python3.12/site-packages/CRISPResso2/CRISPRessoCORE.py", line 3186, in main
    raise CRISPRessoShared.NoReadsAlignedException('Error: No alignments were found')
CRISPResso2.CRISPRessoShared.NoReadsAlignedException: Error: No alignments were found
CRITICAL @ Tue, 10 Jun 2025 09:43:56 (30.0% done):
	 Traceback (most recent call last):
  File "/home/fabad/mambaforge/envs/crispresso2/lib/python3.12/site-packages/CRISPResso2/CRISPRessoCORE.py", line 3186, in main
    raise CRISPRessoShared.NoReadsAlignedException('Error: No alignments were found')
CRISPResso2.CRISPRessoShared.NoReadsAlignedException: Error: No alignments were found
 

CRITICAL @ Tue, 10 Jun 2025 09:43:56 (30.0% done):
	 Alignment error, please check your input.

ERROR: Error: No alignments were found 

However; I have another FASTQ file that does not contain this kind of weird entries and it fails with the WTF error as well. I tried to understand in which conditions the alignment process throw that error, just to identify the cause of the problem, but the algorithm is complex and I was not able to figure out that.