Fix RPMS missing codes being skipped on REDCap imports

[dheshanm]

We noticed subject JE94766 had chrscid_a27 and chrscid_a28 as -3 on RPMS CSVs, but had '3' on REDCap.

Here is what we believe happened:

1. JE94766 had 3 filled in for chrscid_a27 and chrscid_a28 originally, but was later changed to -3
2. rpms_to_redcap.py script skips/ignores missing codes on fields marked @nomissing when uploading to REDCap.
3. So, this updated value is not reflected on REDCap.

We had discussed 2 potential solutions to this issue:

1. Scheduled wiping on add 'PRESCIENT' REDCap records to clean such values
2. Switching to a more restrictive overwriteBehavior on REDCap API:
   • This should overwrite overwrite blank/empty fields as well

https://redcap.partners.org/redcap/api/help/?content=imp_records

[tashrifbillah]

Observation:

• if -3 is pushed up against a radio variable with @NOMISSING tag, the current REDCap (v14.5.17) did not reject that. However, on the front-end, -3 did not have any effect. In other words, the radio options in front-end remained unchecked.

• However, when that JSON was downloaded, -3s showed up in radio variables' field.

---

The above change was possible by replacing continue with value='' in these lines:

utility/rpms_to_redcap.py

Lines 238 to 253 in 5d2777c

	if pd.isna(visit[v]):
	continue
	elif visit[v].lower() in ['-','none','not applicable', 'n/a','na']:
	if ftype=='calc':
	# always reject not applicable
	continue
	elif ftype=='text' and not pd.isna(dtype):
	# if any validation is set, reject not applicable
	continue
	elif visit[v] in ['-3','-9','1903-03-03','1909-09-09'] and annotation=='@NOMISSING':
	continue
	elif visit[v] in ['-3','-9','-99'] and ftype in ['dropdown','yesno','radio','checkbox']:
	# these codes do not fit these field types
	continue

[tashrifbillah]

New issues:

• why did -3 get pushed up when blank value was populated in value? Notably, -3 is present in the original data.