My most used commands and some notes.

Normal use (latest PU, startup conditions, no validation)

Old syntax:
cmsDriver.py TTbar_Tauola_8TeV_cfi.py -s GEN,FASTSIM,HLT:GRun --pileup=2012_Summer_inTimeOnly --geometry DB --conditions=auto:startup_GRun --beamspot Realistic8TeVCollision --eventcontent=FEVTDEBUGHLT --datatier GEN-SIM-DIGI-RECO -n 10 --no_exec

New syntax:
cmsDriver.py TTbar_Tauola_8TeV_cfi.py -s GEN,SIM,RECO,HLT:GRun --fast --pileup=2012_Summer_inTimeOnly --geometry DB --conditions=auto:startup_GRun --beamspot Realistic8TeVCollision --eventcontent=FEVTDEBUGHLT --datatier GEN-SIM-DIGI-RECO -n 10 --no_exec

GIT stuff

To propose modifications:

git cms-addpkg [dir]/[package]
git checkout -b [my-new-branch-name]
...modify...
git add [new files]
git commit -m "message" [modified files]
git push my-cmssw [my-new-branch-name]

Then do pull requests and sign them through the Topic Collector.

My script for quickly checking the FastSim differences between two releases:

wget http://agiamman.web.cern.ch/agiamman/fastsim/produceReleaseNotes.sh

Useful general commands:

List files that differ between this tag and $CMSSW_VERSION:
```
git diff --name-only $CMSSW_VERSION
```
List directories that differ between this tag and $CMSSW_VERSION:
```
git diff --name-only $CMSSW_VERSION | cut -f1,2 -d/ | sort -u
```

Nice graph of changes:

git log --graph --merges --oneline $CMSSW_VERSION..

As above, more detailed:

git log --graph --oneline $CMSSW_VERSION..

To know in which release a certain version of a certain file is contained:
```
git tag --contains [commit hash]
```

Validation

link

Old syntax:

cmsDriver.py SingleMuPt10_cfi.py -s GEN,FASTSIM,HLT,VALIDATION --pileup=NoPileUp --conditions auto:startup_GRun --beamspot Realistic8TeVCollision --eventcontent=FEVTDEBUGHLT --datatier GEN-SIM-DIGI-RECO -n 10
cmsDriver.py step3 -s HARVESTING:validationHarvestingFS --harvesting AtRunEnd --conditions auto:startup --mc --filein file:SingleMuPt10_cfi_py_GEN_FASTSIM_HLT_VALIDATION.root

New syntax:

cmsDriver.py SingleMuPt10_cfi.py -s GEN,SIM,RECO,HLT,VALIDATION --fast --pileup=NoPileUp --conditions auto:startup_GRun --beamspot Realistic8TeVCollision --eventcontent=FEVTDEBUGHLT --datatier GEN-SIM-DIGI-RECO -n 10
cmsDriver.py step3 -s HARVESTING:validationHarvestingFS --harvesting AtRunEnd --conditions auto:startup --mc --filein file:SingleMuPt10_cfi_py_GEN_SIM_RECO_HLT_VALIDATION.root

Comparison tools:

Tracking, official macro: Validation/RecoTrack/test/macro/TrackValHistoPublisher.C (you only need to change the input files - to be taken from here - and MINEFF, MAXEFF, MAXFAKE)
Tracking, private macro for testing iterative tracking: TrackValHistoPublisher.C
Electrons:

produce relval-like events with cmsDriver ZEE_8TeV_cfi --conditions auto:startup_GRun -s GEN,FASTSIM,HLT:GRun,VALIDATION --datatier GEN-SIM-DIGI-RECO -n 10 --relval 100000,500 --eventcontent FEVTDEBUGHLT --no_exec
cvs co Validation/RecoEgamma; scram b
use this private cfg (adapted from Validation/RecoEgamma/test/ElectronMcSignalValidation_cfg.py) to produce a DQM file
use this private comparison macro
reference DQM files can be copied from the EGamma validation pages
script for parameter scan

Jets, private macro: jets.C
Jets, official validation procedure:

cvs co Validation/RecoJets; scram b
cd Validation/RecoJets/analysis
cmsRun allJets_validation_cfg.py

Make sure that a recommended GT is used (link)

Or substitute process.GlobalTag.globaltag = '...' with the following:

from Configuration.AlCa.GlobalTag import GlobalTag
process.GlobalTag = GlobalTag(process.GlobalTag, 'auto:startup_GRun', '')

./validation.sh

enter Reference.root New.root title (normalization by area) or Reference.root New.root title y (normalization by yield)
if it doesn't find the executables, make sure that 1) you compiled 2) the script is assuming the correct architecture

Testing with runTheMatrix (link)

Testing all workflows:
runTheMatrix.py --selected
runTheMatrix.py -i all -j 12 --cafVeto would do everything as in pre-release validation, even more than we run in IBs
runTheMatrix.py -i all -j 12 would run quite similar as in IBs
To test only a specific workflow:
runTheMatrix.py -i all -l 1200.0 >& test.log
(-i all means that GEN-SIM is recycled, which is irrelevant in this specific example)
The workflows are defined here (standard) and here (pileup)
The relval steps are defined here
The FastSim workflows are:
- 5.1 : ttbar
- 5.2 : single mu, pt=10 GeV
- 5.3 : single mu, pt=100 GeV
- 5.4 : Z->ee
- 5.5 : Z->tautau
- 39 : QCD
- 6.4 : H->gammagamma
- 400 : ttbar + old pileup
- 401 : ttbar + new pileup
- 1200 : ttbar, split into GEN-SIM and RECO
- 1201 : ttbar, full sequence with new syntax
When touching the sequences, better to test this combo:
runTheMatrix.py -i all -l 5.1,400.0,401.0,1200.0,1201.0 >& test.log

Note: to checkout a tagset, do

cmstc tagset -a 121587 121610
checkdeps -a -j 50

Customization

--customise_commands="..."

To add time and memory (RSS) profiling:

--customise=Validation/Performance/TimeMemoryInfo.py

Pileup files

Script to produce new minimum-bias files for pileup.
These files must be copied into directory /afs/cern.ch/cms/data/CMSSW/FastSimulation/PileUpProducer/data/
Then remember to update the list in data-FastSimulation-PileUpProducer.spec (*) and to inform the release managers to rebuild data-FastSimulation-PileUpProducer external via a pull request.

(*) fork CMSDIST repository. Clone it locally (see example below), modify the file, push it your forked repository, and make a pull request.

cmsrel CMSSW_7_0_0_preX
cd CMSSW_7_0_0_preX/src
git clone git@github.com:giamman/cmsdist.git cmsdist-giamman
cd cmsdist-giamman
git checkout IB/CMSSW_7_0_X/stable
vim data-FastSimulation-PileUpProducer.spec
git commit -a -s 
# Remember to provide meaningful message.
# Example:
# Removal of X and addition of Y (subject line)
# (empty line)
# X was removed due to ... Y was added instead...  (detailed description of you changes, paragraph). 
# -s option of commit adds your signature to the commit
git push origin HEAD:13Tev-update-data-FastSimulation-PileUpProducer
# Now on GitHub you can make a pull request to official CMSDIST repository

Submission on lxbatch

Script for single-job submission on lxbatch (bsub -q 8nh submit_job.sh)
Remember to make it executable: chmod +x submit_job.sh

If multi-job submission is needed:

download this tar file
untar with tar -zxvf lxbatch.tgz
edit MYWORKDIR and MYOUT in batch.csh
create your config file, then rename it as template.py

add this line: process.RandomNumberGeneratorService.generator.initialSeed = cms.untracked.uint32(123456XXX)
output file name must coincide with batch.csh, for example output.root

edit number of jobs in make_configs.csh and submit_batch.csh, and preferred queue in submit_batch.csh
./make_configs.csh
./submit_batch.csh
NOTE: if the analyzer downstream complains because all files have the same event numbers, just add the line duplicateCheckMode = cms.untracked.string('noDuplicateCheck') in the Source block (example)

Debugging

scram tool info python
then use the value you see for PYTHON_BASE and use it as the value for the environment variable PYTHONHOME
scram build -d
gdb --args cmsRun test.py
catch throw
run
where
if there is more than one exception: continue

Alternatively, to get also the line number:

scram b clean
scram tool info python
then use the value you see for PYTHON_BASE and use it as the value for the environment variable PYTHONHOME
setenv USER_CXXFLAGS "-O0 -g"
scram b -v -k
gdb --args cmsRun test.py
catch throw
run
where
then, after you got it stopped at the right place, do:
up
up
up
list

For the FastSim paper draft

To check out the draft in a fresh directory:

svn co -N svn+ssh://svn.cern.ch/reps/tdr2 myDir # where myDir is a placeholder for a name of your choice
cd myDir
svn update utils
svn update -N papers
svn update papers/GEN-11-001

Then to use the tdr script to build:

eval `./papers/tdr runtime -csh` # for tcsh. use -sh for bash.
cd papers/GEN-11-001/trunk # (edit the template, then to build the document)
tdr --style=paper b GEN-11-001

(or just use ./MAKENOTE)

Add new files with

svn add your_new_file

Update the directory with

svn update

Commit your changes with

svn commit -m "commit message"

Useful material:

Plots by Pedro for BTV-11-002: link
The HEEP ID cuts used by Shilpi are also used in EXO-11-019 (Z') and EXO-10-016 (excited leptons)

To create a branch

cvs rtag -r CMSSW_X_Y_Z -b FAMOS_X_Y_Z Subsystem/Package
cvs co -r FAMOS_X_Y_Z Subsystem/Package
edit file and commit; or import changes with cvs update -j tag1 -j tag2 Subsystem/Package/src/file.cc
cd Subsystem/Package
cvs tag Vxx-yy-zz

Performance plots

Event content dumping

At runtime: add

process.dumpContent = cms.EDAnalyzer('EventContentAnalyzer')

and insert it in some path to see what products are in the event at that point in the process.

Useful links

HCAL noise tuning with neutrino gun

Moved here

HCAL signal tuning with pion gun

Moved here

Correction factors from 500pre6/500pre3 ratio:

###### Ratio vs ieta:
ieta=-40 - ratio = 0.836547
ieta=-39 - ratio = 0.915296
ieta=-38 - ratio = 0.931713
ieta=-37 - ratio = 0.927074
ieta=-36 - ratio = 0.94417
ieta=-35 - ratio = 0.955124
ieta=-34 - ratio = 0.959822
ieta=-33 - ratio = 0.952961
ieta=-32 - ratio = 0.967343
ieta=-31 - ratio = 0.975566
ieta=-30 - ratio = 0.973605
ieta=30 - ratio = 0.979185
ieta=31 - ratio = 0.985501
ieta=32 - ratio = 0.985795
ieta=33 - ratio = 0.993118
ieta=34 - ratio = 0.988925
ieta=35 - ratio = 0.945334
ieta=36 - ratio = 0.963244
ieta=37 - ratio = 0.953914
ieta=38 - ratio = 0.959041
ieta=39 - ratio = 0.916422
ieta=40 - ratio = 0.880503