Objectives:

  • Discuss count normalizations
  • Execute model fitting for differential expression comparisons

1 Differential Expression Workflow

Here we will proceed with count normalizations and fit our DESeq2 model.


2 Count normalizations

Since counts of mapped reads for each gene is proportional to the expression of RNA in addition to many “uninteresting” other factors, normalization is the process of scaling raw count values to account for the “uninteresting” factors and ensure expression levels are more comparable.

2.1 Normalization goals

Two common factors that need to be accounted for during normalization are sequencing depth and gene length. Common normalization approaches (such as FPKM, RPKM, CPM, TPM, etc.) account for one or both of these factors.

  • Sequencing depth normalization is necessary to account for the proportion of reads per gene expected for more deeply sequenced samples (like in pink below) versus a less deeply sequenced sample (like in green below).

Note that each pink or green rectangle represents an aligned read, with reads spanning an intron connected by a dashed line. Figure from HBC training materials

  • Gene length normalization may also be necessary if comparing between different genes, since genes of different lengths have different probabilities of generating fragments that end up in the library.

Note: The above figure was originally from a HBC tutorial that also includes a detailed comparison of different normalization (CPM, TPM, FPKM) approaches and their best uses.

Check-in: Questions about normalizations?

2.2 DESeq2 normalizations

DESeq2 has an internal normalization process that accounts for RNA composition. A few highly differentially expressed genes, differences in the number of genes expressed between samples, or contamination are not accounted for by depth or gene length normalization methods. Accounting for RNA composition is particularly important for differential expression analyses, regardless of the tool used.

For data exploration and visualizations, it is helpful to generate an object of independently normalized counts. We will use the rlog transformation from DESeq2 that accounts for sequencing depth for each sample and RNA composition for the downstream quality control visualizations.

The rlog transformation produces log2 scale data that has also been normalized to overall library size as well as variance across genes at different mean expression levels. For larger numbers of samples, there is an alternative transformation method, vst that can be used instead for count normalizations.

The command to generate the normalized count object has a few parts, including dds as an input and providing a value to the option blind.

rld <- rlog(dds, blind = TRUE)

*Note: We might see a message that our data did not fit the default ‘parametric’ dispersion model so a local regression was substituted. If we had more time, we might look at a dispersion plot with the plotDispEsts(dds) function, but as this bioconductor thread discusses, other visualizations of our data might be more helpful and/or easier to interpret.

Next, we’ll look at the results of the transformation.

head(assay(rld), 2)
##                    Sample_116498 Sample_116499 Sample_116500 Sample_116501 Sample_116502
## ENSMUSG00000000001     12.635215     12.609457      12.76686     12.616921     12.705707
## ENSMUSG00000000028      8.096766      8.350835       8.17923      8.391328      8.616073
##                    Sample_116503 Sample_116504 Sample_116505 Sample_116506 Sample_116507
## ENSMUSG00000000001     12.661095      12.54194      12.52138      12.49388      12.71019
## ENSMUSG00000000028      8.653318      10.71402      10.77955      10.88032      11.01995
##                    Sample_116508 Sample_116509
## ENSMUSG00000000001      12.52632      12.48844
## ENSMUSG00000000028      10.93564      10.96100

Looking at the rld values, we can see that they are now in log scale. Since we set blind=TRUE, the transformation is blind to the sample information we specified in the design formula. The normalized counts are helpful for visualization methods during expression-level quality assessment but aren’t used in the model fitting.

3 DESeq2 Model Fitting

Next, we’ll fit our standard model using the DESeq function and take a look at the objects we generate.

dds <- DESeq(dds)

This command applies the model to our data, using the sample information supplied when generating the dds object so it takes some time to run.

resultsNames(dds)
## [1] "Intercept"                    "Gtype.Tx_ko.Tx_vs_wt.Tx"      "Gtype.Tx_ko.control_vs_wt.Tx"
## [4] "Gtype.Tx_wt.control_vs_wt.Tx"

The results include three pairwise comparisons to the specified control as default, but other information is now stored in the dds object so we can generate additional pairwise comparisons of interest.

head(dds)

Checkpoint: If you see the same results when you execute resultsNames(dds), please indicate with the green ‘yes’ button. Otherwise, please use the red ‘x’ button to get help before the break

3.0.0.1 [Optional Exercise]

If you executed the commands in the option exercise from Module 07, you can fit a seperate DESeq2 model for the patient sample example, which included a covariate in our model.

Click for fitting a model that includes a covariate
dds_patient <- DESeq(dds_patient)
resultsNames(dds_patient)
If you run through the optional exercises, you can explore the impact of adding a covariate by substituting dds_patient for dds and re-running those commands since both DESeq2 objects have their data organized in the same way.




4 Summary

In this section, we:

  • Learned about count normalizations and uses
  • Generated a normalized count table
  • Fit two DESeq2 models for our data
  • Saw the impact of including a covariate in our model

5 Sources

5.1 Training resources used to develop materials

These materials have been adapted and extended from materials listed above. These are open access materials distributed under the terms of the Creative Commons Attribution license (CC BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

LS0tCnRpdGxlOiAiRGF5IDIgLSBNb2R1bGUgMDg6IENvdW50IG5vcm1hbGl6YXRpb25zIGFuZCBERVNlcTIgbW9kZWwgZml0dGluZyIKYXV0aG9yOiAiVU0gQmlvaW5mb3JtYXRpY3MgQ29yZSIKZGF0ZTogImByIFN5cy5EYXRlKClgIgpvdXRwdXQ6CiAgICAgICAgaHRtbF9kb2N1bWVudDoKICAgICAgICAgICAgaW5jbHVkZXM6CiAgICAgICAgICAgICAgICBpbl9oZWFkZXI6IGhlYWRlci5odG1sCiAgICAgICAgICAgIHRoZW1lOiBwYXBlcgogICAgICAgICAgICB0b2M6IHRydWUKICAgICAgICAgICAgdG9jX2RlcHRoOiA0CiAgICAgICAgICAgIHRvY19mbG9hdDogdHJ1ZQogICAgICAgICAgICBudW1iZXJfc2VjdGlvbnM6IHRydWUKICAgICAgICAgICAgZmlnX2NhcHRpb246IHRydWUKICAgICAgICAgICAgbWFya2Rvd246IEdGTQogICAgICAgICAgICBjb2RlX2Rvd25sb2FkOiB0cnVlCi0tLQo8c3R5bGUgdHlwZT0idGV4dC9jc3MiPgpib2R5eyAvKiBOb3JtYWwgICovCiAgICAgIGZvbnQtc2l6ZTogMTRwdDsKICB9CnByZSB7CiAgZm9udC1zaXplOiAxMnB0Cn0KY29kZS5yewogIGZvbnQtc2l6ZTogMTJwdDsKfQo8L3N0eWxlPgoKPCEtLS0gQWxsb3cgdGhlIHBhZ2UgdG8gYmUgd2lkZXIgLS0tPgo8c3R5bGU+CiAgICBib2R5IC5tYWluLWNvbnRhaW5lciB7CiAgICAgICAgbWF4LXdpZHRoOiAxMjAwcHg7CiAgICB9Cjwvc3R5bGU+Cgo+ICMgT2JqZWN0aXZlczogICAgCj4gKiBEaXNjdXNzIGNvdW50IG5vcm1hbGl6YXRpb25zCj4gKiBFeGVjdXRlIG1vZGVsIGZpdHRpbmcgZm9yIGRpZmZlcmVudGlhbCBleHByZXNzaW9uIGNvbXBhcmlzb25zCgpgYGB7ciBMb2FkUnVubmluZ0RhdGEsIGV2YWw9VFJVRSwgZWNobz1GQUxTRSwgbWVzc2FnZT1GQUxTRSwgd2FybmluZz1GQUxTRX0KbGlicmFyeShERVNlcTIpCmxvYWQoInJkYXRhL1J1bm5pbmdEYXRhLlJEYXRhIikKYGBgCgoKIyBEaWZmZXJlbnRpYWwgRXhwcmVzc2lvbiBXb3JrZmxvdwoKSGVyZSB3ZSB3aWxsIHByb2NlZWQgd2l0aCBjb3VudCBub3JtYWxpemF0aW9ucyBhbmQgZml0IG91ciBERVNlcTIgbW9kZWwuCgohW10oLi9pbWFnZXMvd2F5ZmluZGVyL3dheWZpbmRlci0wNC5wbmcpe3dpZHRoPTc1JX0KCi0tLQoKIyBDb3VudCBub3JtYWxpemF0aW9ucwoKU2luY2UgY291bnRzIG9mIG1hcHBlZCByZWFkcyBmb3IgZWFjaCBnZW5lIGlzIHByb3BvcnRpb25hbCB0byB0aGUgZXhwcmVzc2lvbiBvZiBSTkEgaW4gYWRkaXRpb24gdG8gbWFueSDigJx1bmludGVyZXN0aW5n4oCdIG90aGVyIGZhY3RvcnMsIG5vcm1hbGl6YXRpb24gaXMgdGhlIHByb2Nlc3Mgb2Ygc2NhbGluZyByYXcgY291bnQgdmFsdWVzIHRvIGFjY291bnQgZm9yIHRoZSDigJx1bmludGVyZXN0aW5n4oCdIGZhY3RvcnMgYW5kIGVuc3VyZSBleHByZXNzaW9uIGxldmVscyBhcmUgbW9yZSBjb21wYXJhYmxlLgoKIyMgTm9ybWFsaXphdGlvbiBnb2FscwoKVHdvIGNvbW1vbiBmYWN0b3JzIHRoYXQgbmVlZCB0byBiZSBhY2NvdW50ZWQgZm9yIGR1cmluZyBub3JtYWxpemF0aW9uIGFyZSAqKnNlcXVlbmNpbmcgZGVwdGgqKiBhbmQgKipnZW5lIGxlbmd0aCoqLiBDb21tb24gbm9ybWFsaXphdGlvbiBhcHByb2FjaGVzIChzdWNoIGFzIEZQS00sIFJQS00sIENQTSwgVFBNLCBldGMuKSBhY2NvdW50IGZvciBvbmUgb3IgYm90aCBvZiB0aGVzZSBmYWN0b3JzLiAgICAKCiogKipTZXF1ZW5jaW5nIGRlcHRoKiogbm9ybWFsaXphdGlvbiBpcyBuZWNlc3NhcnkgdG8gYWNjb3VudCBmb3IgdGhlIHByb3BvcnRpb24gb2YgcmVhZHMgcGVyIGdlbmUgZXhwZWN0ZWQgZm9yIG1vcmUgZGVlcGx5IHNlcXVlbmNlZCBzYW1wbGVzIChsaWtlIGluIHBpbmsgYmVsb3cpIHZlcnN1cyBhIGxlc3MgZGVlcGx5IHNlcXVlbmNlZCBzYW1wbGUgKGxpa2UgaW4gZ3JlZW4gYmVsb3cpLiAgICAKCiFbKk5vdGUgdGhhdCBlYWNoIHBpbmsgb3IgZ3JlZW4gcmVjdGFuZ2xlIHJlcHJlc2VudHMgYW4gYWxpZ25lZCByZWFkLCB3aXRoIHJlYWRzIHNwYW5uaW5nIGFuIGludHJvbiBjb25uZWN0ZWQgYnkgYSBkYXNoZWQgbGluZS4gRmlndXJlIGZyb20gW0hCQyAgdHJhaW5pbmcgbWF0ZXJpYWxzXShodHRwczovL2hiY3RyYWluaW5nLmdpdGh1Yi5pby9ER0Vfd29ya3Nob3AvbGVzc29ucy8wMl9ER0VfY291bnRfbm9ybWFsaXphdGlvbi5odG1sKSpdKC4vaW1hZ2VzL25vcm1hbGl6YXRpb25fbWV0aG9kc19kZXB0aC5wbmcpICAgIAoKCiogKipHZW5lIGxlbmd0aCoqIG5vcm1hbGl6YXRpb24gbWF5IGFsc28gYmUgbmVjZXNzYXJ5IGlmIGNvbXBhcmluZyBiZXR3ZWVuIGRpZmZlcmVudCBnZW5lcywgc2luY2UgZ2VuZXMgb2YgZGlmZmVyZW50IGxlbmd0aHMgaGF2ZSBkaWZmZXJlbnQgcHJvYmFiaWxpdGllcyBvZiBnZW5lcmF0aW5nIGZyYWdtZW50cyB0aGF0IGVuZCB1cCBpbiB0aGUgbGlicmFyeS4KCgo+ICoqTm90ZSoqOiBUaGUgYWJvdmUgZmlndXJlIHdhcyBvcmlnaW5hbGx5IGZyb20gYSBbSEJDICB0dXRvcmlhbF0oaHR0cHM6Ly9oYmN0cmFpbmluZy5naXRodWIuaW8vREdFX3dvcmtzaG9wL2xlc3NvbnMvMDJfREdFX2NvdW50X25vcm1hbGl6YXRpb24uaHRtbCkgdGhhdCBhbHNvIGluY2x1ZGVzIGEgZGV0YWlsZWQgY29tcGFyaXNvbiBvZiBkaWZmZXJlbnQgbm9ybWFsaXphdGlvbiAoQ1BNLCBUUE0sIEZQS00pIGFwcHJvYWNoZXMgYW5kIHRoZWlyIGJlc3QgdXNlcy4KCioqQ2hlY2staW46IFF1ZXN0aW9ucyBhYm91dCBub3JtYWxpemF0aW9ucz8qKgoKIyMgREVTZXEyIG5vcm1hbGl6YXRpb25zCgpERVNlcTIgaGFzIGFuIFtpbnRlcm5hbCBub3JtYWxpemF0aW9uIHByb2Nlc3NdKGh0dHBzOi8vZ2Vub21lYmlvbG9neS5iaW9tZWRjZW50cmFsLmNvbS9hcnRpY2xlcy8xMC4xMTg2L2diLTIwMTAtMTEtMTAtcjEwNikgdGhhdCBhY2NvdW50cyBmb3IgKipSTkEgY29tcG9zaXRpb24qKi4gQSBmZXcgaGlnaGx5IGRpZmZlcmVudGlhbGx5IGV4cHJlc3NlZCBnZW5lcywgZGlmZmVyZW5jZXMgaW4gdGhlIG51bWJlciBvZiBnZW5lcyBleHByZXNzZWQgYmV0d2VlbiBzYW1wbGVzLCBvciBjb250YW1pbmF0aW9uIGFyZSBub3QgYWNjb3VudGVkIGZvciBieSBkZXB0aCBvciBnZW5lIGxlbmd0aCBub3JtYWxpemF0aW9uIG1ldGhvZHMuIEFjY291bnRpbmcgZm9yIFJOQSBjb21wb3NpdGlvbiBpcyBwYXJ0aWN1bGFybHkgaW1wb3J0YW50IGZvciBkaWZmZXJlbnRpYWwgZXhwcmVzc2lvbiBhbmFseXNlcywgcmVnYXJkbGVzcyBvZiB0aGUgdG9vbCB1c2VkLgoKRm9yIGRhdGEgZXhwbG9yYXRpb24gYW5kIHZpc3VhbGl6YXRpb25zLCBpdCBpcyBoZWxwZnVsIHRvIGdlbmVyYXRlIGFuIG9iamVjdCBvZiBpbmRlcGVuZGVudGx5IG5vcm1hbGl6ZWQgY291bnRzLiBXZSB3aWxsIHVzZSB0aGUgW3Jsb2cgdHJhbnNmb3JtYXRpb25dKGh0dHA6Ly9iaW9jb25kdWN0b3Iub3JnL3BhY2thZ2VzL2RldmVsL2Jpb2MvdmlnbmV0dGVzL0RFU2VxMi9pbnN0L2RvYy9ERVNlcTIuaHRtbCNjb3VudC1kYXRhLXRyYW5zZm9ybWF0aW9ucykgZnJvbSBERVNlcTIgdGhhdCBhY2NvdW50cyBmb3Igc2VxdWVuY2luZyBkZXB0aCBmb3IgZWFjaCBzYW1wbGUgYW5kIFJOQSBjb21wb3NpdGlvbiBmb3IgdGhlIGRvd25zdHJlYW0gcXVhbGl0eSBjb250cm9sIHZpc3VhbGl6YXRpb25zLgoKVGhlIHJsb2cgdHJhbnNmb3JtYXRpb24gcHJvZHVjZXMgbG9nMiBzY2FsZSBkYXRhIHRoYXQgaGFzIGFsc28gYmVlbiBub3JtYWxpemVkIHRvIG92ZXJhbGwgbGlicmFyeSBzaXplIGFzIHdlbGwgYXMgdmFyaWFuY2UgYWNyb3NzIGdlbmVzIGF0IGRpZmZlcmVudCBtZWFuIGV4cHJlc3Npb24gbGV2ZWxzLiBGb3IgbGFyZ2VyIG51bWJlcnMgb2Ygc2FtcGxlcywgdGhlcmUgaXMgYW4gYWx0ZXJuYXRpdmUgdHJhbnNmb3JtYXRpb24gbWV0aG9kLCBbdnN0XShodHRwOi8vd3d3LmJpb2NvbmR1Y3Rvci5vcmcvcGFja2FnZXMvcmVsZWFzZS9iaW9jL3ZpZ25ldHRlcy9ERVNlcTIvaW5zdC9kb2MvREVTZXEyLmh0bWwjY291bnQtZGF0YS10cmFuc2Zvcm1hdGlvbnMpIHRoYXQgY2FuIGJlIHVzZWQgaW5zdGVhZCBmb3IgY291bnQgbm9ybWFsaXphdGlvbnMuCgoKVGhlIGNvbW1hbmQgdG8gZ2VuZXJhdGUgdGhlIG5vcm1hbGl6ZWQgY291bnQgb2JqZWN0IGhhcyBhIGZldyBwYXJ0cywgaW5jbHVkaW5nIGBkZHNgIGFzIGFuIGlucHV0IGFuZCBwcm92aWRpbmcgYSB2YWx1ZSB0byB0aGUgb3B0aW9uIGBibGluZGAuCmBgYHtyIENvdW50Tm9ybX0KcmxkIDwtIHJsb2coZGRzLCBibGluZCA9IFRSVUUpCmBgYAoKKk5vdGU6IFdlIG1pZ2h0IHNlZSBhIG1lc3NhZ2UgdGhhdCBvdXIgZGF0YSBkaWQgbm90IGZpdCB0aGUgZGVmYXVsdCAncGFyYW1ldHJpYycgZGlzcGVyc2lvbiBtb2RlbCBzbyBhIGxvY2FsIHJlZ3Jlc3Npb24gd2FzIHN1YnN0aXR1dGVkLiBJZiB3ZSBoYWQgbW9yZSB0aW1lLCB3ZSBtaWdodCBsb29rIGF0IGEgZGlzcGVyc2lvbiBwbG90IHdpdGggdGhlIGBwbG90RGlzcEVzdHMoZGRzKWAgZnVuY3Rpb24sIGJ1dCBhcyBbdGhpcyBiaW9jb25kdWN0b3IgdGhyZWFkXShodHRwczovL3N1cHBvcnQuYmlvY29uZHVjdG9yLm9yZy9wLzEwNzkzNy8pIGRpc2N1c3Nlcywgb3RoZXIgdmlzdWFsaXphdGlvbnMgb2Ygb3VyIGRhdGEgbWlnaHQgYmUgbW9yZSBoZWxwZnVsIGFuZC9vciBlYXNpZXIgdG8gaW50ZXJwcmV0LgoKTmV4dCwgd2UnbGwgbG9vayBhdCB0aGUgcmVzdWx0cyBvZiB0aGUgdHJhbnNmb3JtYXRpb24uCmBgYHtyIENvdW50Tm9ybUNoZWNrfQpoZWFkKGFzc2F5KHJsZCksIDIpCmBgYAoKTG9va2luZyBhdCB0aGUgcmxkIHZhbHVlcywgd2UgY2FuIHNlZSB0aGF0IHRoZXkgYXJlIG5vdyBpbiBsb2cgc2NhbGUuIFNpbmNlIHdlIHNldCBgYmxpbmQ9VFJVRWAsIHRoZSB0cmFuc2Zvcm1hdGlvbiBpcyBibGluZCB0byB0aGUgc2FtcGxlIGluZm9ybWF0aW9uIHdlIHNwZWNpZmllZCBpbiB0aGUgZGVzaWduIGZvcm11bGEuIFRoZSBub3JtYWxpemVkIGNvdW50cyBhcmUgaGVscGZ1bCBmb3IgdmlzdWFsaXphdGlvbiBtZXRob2RzIGR1cmluZyBleHByZXNzaW9uLWxldmVsIHF1YWxpdHkgYXNzZXNzbWVudCBidXQgKiphcmVuJ3QgdXNlZCBpbiB0aGUgbW9kZWwgZml0dGluZyoqLgoKIyBERVNlcTIgTW9kZWwgRml0dGluZwoKTmV4dCwgd2UnbGwgZml0IG91ciBzdGFuZGFyZCBtb2RlbCB1c2luZyB0aGUgYERFU2VxYCBmdW5jdGlvbiBhbmQgdGFrZSBhIGxvb2sgYXQgdGhlIG9iamVjdHMgd2UgZ2VuZXJhdGUuCmBgYHtyIEZpdE1vZGVsU3RhbmRhcmQsIG1lc3NhZ2U9RkFMU0V9CmRkcyA8LSBERVNlcShkZHMpCmBgYAoKVGhpcyBjb21tYW5kIGFwcGxpZXMgdGhlIG1vZGVsIHRvIG91ciBkYXRhLCB1c2luZyB0aGUgc2FtcGxlIGluZm9ybWF0aW9uIHN1cHBsaWVkIHdoZW4gZ2VuZXJhdGluZyB0aGUgYGRkc2Agb2JqZWN0IHNvIGl0IHRha2VzIHNvbWUgdGltZSB0byBydW4uCgpgYGB7ciBGaXRNb2RlbFN0YW5kYXJkQ2hlY2sxLCBldmFsPVRSVUV9CnJlc3VsdHNOYW1lcyhkZHMpCmBgYAoKVGhlIHJlc3VsdHMgaW5jbHVkZSB0aHJlZSBwYWlyd2lzZSBjb21wYXJpc29ucyB0byB0aGUgc3BlY2lmaWVkIGNvbnRyb2wgYXMgZGVmYXVsdCwgYnV0IG90aGVyIGluZm9ybWF0aW9uIGlzIG5vdyBzdG9yZWQgaW4gdGhlIGBkZHNgIG9iamVjdCBzbyB3ZSBjYW4gZ2VuZXJhdGUgYWRkaXRpb25hbCBwYWlyd2lzZSBjb21wYXJpc29ucyBvZiBpbnRlcmVzdC4KCmBgYHtyIEZpdE1vZGVsU3RhbmRhcmRDaGVjazIsIGV2YWw9RkFMU0V9CmhlYWQoZGRzKQpgYGAKCioqQ2hlY2twb2ludCoqOiAqSWYgeW91IHNlZSB0aGUgc2FtZSByZXN1bHRzIHdoZW4geW91IGV4ZWN1dGUgYHJlc3VsdHNOYW1lcyhkZHMpYCwgcGxlYXNlIGluZGljYXRlIHdpdGggdGhlIGdyZWVuICd5ZXMnIGJ1dHRvbi4gT3RoZXJ3aXNlLCBwbGVhc2UgdXNlIHRoZSByZWQgJ3gnIGJ1dHRvbiB0byBnZXQgaGVscCBiZWZvcmUgdGhlIGJyZWFrKgoKCgojIyMjIFtPcHRpb25hbCBFeGVyY2lzZV0KCklmIHlvdSBleGVjdXRlZCB0aGUgY29tbWFuZHMgaW4gdGhlIG9wdGlvbiBleGVyY2lzZSBmcm9tIE1vZHVsZSAwNywgeW91IGNhbiBmaXQgYSBzZXBlcmF0ZSBERVNlcTIgbW9kZWwgZm9yIHRoZSBwYXRpZW50IHNhbXBsZSBleGFtcGxlLCB3aGljaCBpbmNsdWRlZCBhIGNvdmFyaWF0ZSBpbiBvdXIgbW9kZWwuCgo8ZGV0YWlscz4KICAgIDxzdW1tYXJ5PipDbGljayBmb3IgZml0dGluZyBhIG1vZGVsIHRoYXQgaW5jbHVkZXMgYSBjb3ZhcmlhdGUqPC9zdW1tYXJ5PgpgYGB7ciBGaXRNb2RlbENvdmFyaWF0ZSwgZXZhbD0gRkFMU0UsIG1lc3NhZ2U9RkFMU0V9CmRkc19wYXRpZW50IDwtIERFU2VxKGRkc19wYXRpZW50KQpgYGAKYGBge3IgRml0TW9kZWxDb3ZhcmlhdGVDaGVjazEsZXZhbD1GQUxTRX0KcmVzdWx0c05hbWVzKGRkc19wYXRpZW50KQpgYGAKICAgIElmIHlvdSBydW4gdGhyb3VnaCB0aGUgb3B0aW9uYWwgZXhlcmNpc2VzLCB5b3UgY2FuIGV4cGxvcmUgdGhlIGltcGFjdCBvZiBhZGRpbmcgYSBjb3ZhcmlhdGUgYnkgc3Vic3RpdHV0aW5nIGBkZHNfcGF0aWVudGAgZm9yIGBkZHNgIGFuZCByZS1ydW5uaW5nIHRob3NlIGNvbW1hbmRzIHNpbmNlIGJvdGggREVTZXEyIG9iamVjdHMgaGF2ZSB0aGVpciBkYXRhIG9yZ2FuaXplZCBpbiB0aGUgc2FtZSB3YXkuCjwvZGV0YWlscz4KCjxicj4KPGJyPgo8YnI+CgoKIyBTdW1tYXJ5CgpJbiB0aGlzIHNlY3Rpb24sIHdlOiAgIAoKKiBMZWFybmVkIGFib3V0IGNvdW50IG5vcm1hbGl6YXRpb25zIGFuZCB1c2VzICAgIAoqIEdlbmVyYXRlZCBhIG5vcm1hbGl6ZWQgY291bnQgdGFibGUgICAgCiogRml0IHR3byBERVNlcTIgbW9kZWxzIGZvciBvdXIgZGF0YSAgICAKKiBTYXcgdGhlIGltcGFjdCBvZiBpbmNsdWRpbmcgYSBjb3ZhcmlhdGUgaW4gb3VyIG1vZGVsCgoKCi0tLQoKIyBTb3VyY2VzCiMjIFRyYWluaW5nIHJlc291cmNlcyB1c2VkIHRvIGRldmVsb3AgbWF0ZXJpYWxzCiogSEJDIERHRSBzZXR1cDogaHR0cHM6Ly9oYmN0cmFpbmluZy5naXRodWIuaW8vREdFX3dvcmtzaG9wL2xlc3NvbnMvMDFfREdFX3NldHVwX2FuZF9vdmVydmlldy5odG1sCiogSEJDIENvdW50IE5vcm1hbGl6YXRpb246IGh0dHBzOi8vaGJjdHJhaW5pbmcuZ2l0aHViLmlvL0RHRV93b3Jrc2hvcC9sZXNzb25zLzAyX0RHRV9jb3VudF9ub3JtYWxpemF0aW9uLmh0bWwKKiBERVNlcTIgc3RhbmRhcmQgdmlnbmV0dGU6IGh0dHA6Ly9iaW9jb25kdWN0b3Iub3JnL3BhY2thZ2VzL2RldmVsL2Jpb2MvdmlnbmV0dGVzL0RFU2VxMi9pbnN0L2RvYy9ERVNlcTIuaHRtbAoqIERFU2VxMiBiZWdpbm5lcnMgdmlnbmV0dGU6IGh0dHBzOi8vYmlvYy5pc20uYWMuanAvcGFja2FnZXMvMi4xNC9iaW9jL3ZpZ25ldHRlcy9ERVNlcTIvaW5zdC9kb2MvYmVnaW5uZXIucGRmCiogQmlvY29uZHVjdG9yIFJOQS1zZXEgV29ya2Zsb3dzOiBodHRwczovL3d3dy5iaW9jb25kdWN0b3Iub3JnL2hlbHAvY291cnNlLW1hdGVyaWFscy8yMDE1L0xlYXJuQmlvY29uZHVjdG9yRmViMjAxNS9CMDIuMV9STkFTZXEuaHRtbAoKCgpgYGB7ciBXcml0ZU91dC5SRGF0YSwgZXZhbD1UUlVFLCBlY2hvPUZBTFNFLCBtZXNzYWdlPUZBTFNFLCB3YXJuaW5nPUZBTFNFfQojSGlkZGVuIGNvZGUgYmxvY2sgdG8gd3JpdGUgb3V0IGRhdGEgZm9yIGtuaXR0aW5nCnNhdmUuaW1hZ2UoZmlsZSA9ICJyZGF0YS9SdW5uaW5nRGF0YS5SRGF0YSIpCgpgYGAKCgpUaGVzZSBtYXRlcmlhbHMgaGF2ZSBiZWVuIGFkYXB0ZWQgYW5kIGV4dGVuZGVkIGZyb20gbWF0ZXJpYWxzIGxpc3RlZCBhYm92ZS4gVGhlc2UgYXJlIG9wZW4gYWNjZXNzIG1hdGVyaWFscyBkaXN0cmlidXRlZCB1bmRlciB0aGUgdGVybXMgb2YgdGhlIFtDcmVhdGl2ZSBDb21tb25zIEF0dHJpYnV0aW9uIGxpY2Vuc2UgKENDIEJZIDQuMCldKGh0dHA6Ly9jcmVhdGl2ZWNvbW1vbnMub3JnL2xpY2Vuc2VzL2J5LzQuMC8pLCB3aGljaCBwZXJtaXRzIHVucmVzdHJpY3RlZCB1c2UsIGRpc3RyaWJ1dGlvbiwgYW5kIHJlcHJvZHVjdGlvbiBpbiBhbnkgbWVkaXVtLCBwcm92aWRlZCB0aGUgb3JpZ2luYWwgYXV0aG9yIGFuZCBzb3VyY2UgYXJlIGNyZWRpdGVkLgo=