This HTML page was converted from the original PDF. Some images and complex layouts may not appear correctly. For the best experience, please download the PDF above.
Python in Excel
A Field Guide
Full Campaign Build with xlwings Lite & AI
Module 03
- Waterfall
- Segmentation
- Stratified Test & Control
- Statistical Tests
- Segment Profiles & Audit Reports
Where the Rubber Hits the Road
Lessons from 25+ years in the trenches
analyzing data, building ML models and
executing campaigns
Campaign + Simple Rules = Money
Campaign + Model = More Money
Campaign + Model + List Error = Disaster
Do the Math
This is business reality
The Campaign Build
- Run Waterfall
- Identify eligible pop
- Review waterfall and audit reports
- Segment Leads
- Create tiered segments
- Review segment profile report
- Set up Control Group
- Setup control group - stratified sampling
- Run statistical tests for consistency of cuts
- Review profile and audit reports
- Setup Test Cells
- Setup test cells for offer or A/B testing
- Review profile and test reports
This is the core process I've used for numerous successful campaigns. Earlier hand-coded, now AI-assisted. And very recently used for an outbound campaign for a SMB: response model built in a Notebook, and the campaign set up in xlwings Lite.
xlwings Lite for Campaigns
![Screenshot of xlwings Lite interface showing Python code editor with imports and a green button labeled "apply_exclusions".]
- Python code, logs, criteria, output - everything in one place
- Data set for each critical step as separate sheet
- Profile reports and statistical tests
The result: a single workbook that serves as both a workspace and a permanent, auditable trail
My Protocol for AI Code Generation
Show what you see
Provide full context - images, sample rows, schemas,
business rules, examples, documentation, gotchas
Tell What You Want
And how you want it done. One thing at a time. In
real life, one instruction not enough - iterate
Inspect What you Expect
Validations are sacrosanct. AI or no AI
AI needs guidance. It makes mistakes
Getting Started
Overview of Process. Details in following pages
Step 1: Download Module 03 Kit
[app.tigzig.com])(undefined) -> xlwings Practice Lab
- AI Coder Instruction Markdown File
- Practice Workbook & Completed Workbook
- Guide : This document
All AI instructions available in the workbooks
Step 2 : Go to [aistudio.google.com])(undefined)
Step 3 : Upload AI Instructions into chat box
- Upload AI Coder instructions
- Copy Paste Base Instruction and hit run
AI is now ready for xlwings Lite code generation
Step 4 Onwards
- Copy paste instructions for code generation
- Check AI understanding & Run code
- Validate results and iterate
Download Module-03 Kit
Go to [app.tigzig.com])(undefined)
xlwings Lite: Practice Lab
Learn xlwings Lite by practical examples
- AI Coder Instructions
For AI
Download the complete markdown file. Upload to your AI at start of every session.
[⬇ Download]
- Workbooks and Guide
Module-03: Campaign Build |
|
A step-by-step guide for working with AI to execute a full, auditable campaign workflow in xlwings Lite |
What’s AI Coder Instruction File
- This is my personal, 1,855-line rulebook for xlwings Lite that I feed to the AI right at the start of every session
- Covers non-negotiable rules, error patterns, tool limitations, and best practices from my client projects
- Helps in delivering clean, reliable xlwings Lite code - reducing debugging and iterations.
3. AI Coder Checklist: Core Directives for All Scripts
Golden Rules: These 20 directives are non-negotiable and MUST be applied in every script.
- Working with Web APIs
- Connecting to Databases via an API Layer
- Python Dependencies Management
- Env. Variables / CORS
- Limitations
Example Scripts
- Starter Examples
- Database Integration
- Web Scraping with LLM Processing
- EDA and Schema Analysis with LLMs
- XGBoost Response Model
- Credit Card Segment Analysis
Putting the Tools to Work
Go to [aistudio.google.com])(undefined)
Welcome to AI Studio
I'm working with xlwings Lite, the browser-based Excel Python add-in released in Jan 2025. It builds on the classic xlwings API but has key differences and limitations. I've attached a detailed reference document that should be treated as the primary source - it includes instructions, known API gaps, workarounds, troubleshooting notes, and working code examples specific to xlwings Lite (scripts, custom functions, FastAPI integrations, etc.). When helping me with code generation or debugging, always refer to this document first. If something seems unsupported or behaves differently from regular xlwings, assume the difference is documented - search the file carefully before answering.
AI_CODER_INSTRUCTIONS.md 15,503 tokens
- Upload AI Coder Instructions File
- Enter Instruction -01 : Base Instruction
(Available in downloaded workbooks) - Hit Run.
AI is now calibrated and ready for xlwings Lite coding work
Data - Sample Rows
4274874 | 4264887 | 4279904 | 4262941 | |
---|---|---|---|---|
CARD_TYPE | SILVER | SILVER | SILVER | SILVER |
BLOCK_CODE | P | |||
BEHAVIOUR_SCORE | 604 | 607 | 630 | 611 |
DNC_TAG | 1 | |||
EVER_30P_P12M | 1 | |||
DPD_CURRENT | ||||
INQUIRY_CASH_P15 | 5 | |||
CASH_PROPENSITY_DECILE | 5 | 5 | ||
PROPENSITY_SCORE | 533 | 538 | ||
CREDIT_LIMIT | 45,000 | 20,000 | 75,000 | 55,000 |
UTIL_CURRENT | 0% | 37% | 43% | 0% |
CURR_BAL | - | 7,425 | 31,924 | - |
OTB_CURRENT | 45,000 | 12,575 | 43,076 | 55,000 |
Data set: 25000 rows
Note: This is a synthetic dataset, purpose-built to test this workflow.
Complex variable correlations may not hold as they would in a live portfolio.
Part 1 : Running the Waterfall
The waterfall criteria / exclusion logic
BUSINESS RULE | CODE LOGIC |
---|---|
Exclude blocked | Drop all non blanks |
Exclude current DPD | Drop where current DPD >= 3 days |
Exclude ever 30+ DPD in past 12 months | Drop ever 30+ DPD >= 1 |
Exclude Behaviour Score <600 | Drop where B score <600 |
Exclude >=90% util with min loan | Drop where ( CURR_BAL + Min. loan of 10000 ) >= 90% of Credit Limit |
Exclude all Do Not Call | Drop where DNC =1 |
Example Criteria: Outbound Personal Loan Campaign on a Credit Card Portfolio (Loan against OTB)
Instruct AI to run waterfall
Instruction 02: Run waterfall and extract eligible leads
Objective is to run a waterfall and get the number of eligible leads and waterfall report.
Exclusion criteria shared below. Apply criteria in that exact sequence only. Data is in table = CUST.
- Add a new field called REASON_CODE from the criteria tables and tag remaining customers with reason code 7.ELIGIBLES.
- Put this data into a table called CUST2 in a new sheet called CUST2.
Extract all the eligibles and put the output into a new sheet and table, both named LEADS. Delete existing sheet and table, if any.
Create a Waterfall report: The waterfall report is to be in the format shared below. Use the LEADS table.
Put that into a new sheet called WF_REP. Delete existing, if any, before updating.
Share your understanding of the criteria, overall requirement as well as the pseudo code.
Share runnable script with the function named as A_RUN_WATERFALL
Instruct what you want done and how you want it done - be specific
ALWAYS ask for AI understanding and pseudo code
Sharing below sample records, waterfall criteria and
Power User Tip : Dictate
Your edge is in expressing your requirements and business logic to AI and validating results - not your typing speed. Voice is the fastest way to transfer that logic to AI.
The Instructions in This Guide Are Sanitized
The instructions here are edited for clarity. They don't reflect how I actually work day to day.
In the Real World, I Dictate - Not Type
Some of my instructions are long and technically complex. For anything longer than a single sentences, I use voice typing. It’s faster and frictionless.
Windows: Win + H
macOS: Press Fn key twice
Focus on Logic, Not Grammar
Dictated instructions often have typos and ah, ooh and pauses That’s fine. AI understands intent
Send out the instruction
Copy paste instruction into aistudio and hit run
Chat Prompt
I am calibrated and ready to assist you as your specialized xlwings Lite AI Coder. Please share your first instruction or code for review.
Objective is to run a waterfall and get the number of eligible leads and waterfall report.
[Exclusion criteria shared below. Apply criteria...]
AI will share 3 things with you ->
1: AI Confirms its Understanding
Understanding of the Criteria
First, here is my understanding of the waterfall exclusion logic based on the image and your sample data. The criteria will be applied in the exact sequence provided, and a customer will be tagged with the first reason they are excluded for:
- 1. BLOCKED: Exclude any customer where the
BLOCK_CODE
is not empty. - 2. DPD_CURR: Of the remaining customers, exclude those where
DPD_CURRENT
is 3 or more. - 3. DPD_EVER: Of the remaining, exclude those where
EVER_30P_P12M
(ever 30+ Days Past Due) is 1 or more. - 4. BSCORE: Of the remaining, exclude those where
BEHAVIOUR_SCORE
is less than [text cut off].
This is your first validation checkpoint. AI translates your request into plain English. Does this perfectly match your business logic? If there are any differences, correct AI now.
2: AI Outlines the Plan
Pseudo-code
Here is the high-level plan for the script:
Setup:
- Load the data from the
CUST
Excel table into a pandas DataFrame. - Delete the
CUST2
andLEADS
sheets if they already exist to ensure a clean run. - Add a new
REASON_CODE
column to the DataFrame, initially empty.
- Load the data from the
Waterfall Exclusion:
- Iterate through the exclusion rules in sequence.
- For each rule, create a boolean mask to identify customers who meet the exclusion criteria and have not yet been assigned a
REASON_CODE
. - Update the
REASON_CODE
column for the newly excluded customers. - Print the count of customers excluded at each step.
Tag Eligible Leads:
This is a logic review. Pseudocode isn't code; it's a step-by-step plan in English. This is where you catch potential flaws before they become bugs in the script. Especially critical to review if you are new to Python.
3: AI Generates the Script
Python Script for xlwings Lite
Please copy and paste the code below into the main.py
tab of your xlwings Lite task pane.
Generated python
import xlwings as xw
from xlwings import script
import pandas as pd
# THIS HELPER FUNCTION IS MANDATORY FOR ALL SCRIPTS ACCESSING TABLES
def find_table_in_workbook(book: xw.Book, table_name: str):
pass
This is the final deliverable for this step. Because you've already validated the understanding and the logic, you can have high confidence on the script. Copy this into main.py
tab on xlwings Lite and run the code.
Run Code
Ribbon menu items visible:
- Sort & Filter
- Find & Select
- Add-ins
- Analyze Data
- Copilot
- Excel Labs
- xlwings Lite
xlwings Lite
import xlwings as xw
from xlwings import script
import pandas as pd
# THIS HELPER FUNCTION IS MANDATORY FOR ALL SCRIPTS ACCESSED
def find_table_in_workbook(book: xw.Book, table_name: str):
"""
Searches all sheets for a table and returns both the
Returns: (xw.Sheet, xw.Table) or (None, None)
"""
Want to dig deeper into the code? Have AI walk you through each code element, step-by-step.
Review Logs & Errors
Review logs, output or xlwings Lite windows for any errors. Copy paste the error log, output or screenshot to AI - it will fix it for you.
def clean_phone_numbers(book: xw.Book):
"""
and adds a new 'CLEANED_PHONE' column with the results
"""
print("🚀 Starting phone number cleaning script...")
try:
# writing cleaned data back to Excel...
# Success! A 'CLEANED_PHONE' column has been added to the PHONES table
# Starting phone number cleaning script...
# Searching for table 'PHONES' in all sheets...
# Found table 'PHONES' on sheet 'DATA'
# Reading data from the 'PHONES' table into DataFrame
# Processing phone numbers...
# Writing cleaned data back to Excel...
# Success! A 'CLEANED_PHONE' column has been added to the PHONES table
# Starting phone number cleaning script...
# Searching for table 'PHONESS' in all sheets...
# Table 'PHONESS' not found in the workbook.
# An error occurred: Could not proceed because table 'PHONES' was not found
# Starting phone number cleaning script...
# Searching for table 'PHONESS' in all sheets...
# Table 'PHONESS' not found in the workbook.
# An error occurred: Could not proceed because table 'PHONES' was not found
xlwings Lite
InvalidArgument: The argument is invalid or missing or has an incorrect format.
raise NotImplementedError()
NotImplementedError
Some errors might take more than one iteration to fix.
Check output and logs and share feedback with AI.
Lists & waterfall report are ready
- CUST2: Full list with additional fields for validations
- LEADS: Eligibles list for further processing
UTIL_CURRENT | CURR_BAL | OTB_CURRENT | REASON_CODE |
---|---|---|---|
0.50 | 14,932.61 | 15,067.39 | 7.ELIGIBLES |
- | - | 105,000.00 | 7.ELIGIBLES |
0.41 | 118,268.41 | 166,731.59 | 7.ELIGIBLES |
0.30 | 28,426.61 | 66,573.39 | 7.ELIGIBLES |
0.49 | 14,648.28 | 15,351.72 | 7.ELIGIBLES |
- | - | 45,000.00 | 7.ELIGIBLES |
0.43 | 30,367.69 | 39,632.31 | 7.ELIGIBLES |
- | - | 60,000.00 | 7.ELIGIBLES |
- | - | 25,000.00 | 7.ELIGIBLES |
0.37 | 7,463.66 | 12,536.34 | 7.ELIGIBLES |
Review list with filters and pivots, in addition to waterfall audit report
Review Waterfall Report
REASON_CODE | DESC | DROPS | REMAINING |
---|---|---|---|
Total Customers | 25,000 | ||
1. BLOCKED | Exclude blocked | 5,012 | 19,988 |
2. DPD_CURR | Exclude current DPD | 385 | 19,603 |
3. DPD_EVER | Exclude ever 30+ DPD P 12M | 1,993 | 17,610 |
4. BSCORE | Exclude Behaviour Score <600 | 838 | 16,772 |
5. NO_OTB | Exclude >=90% util with min loan | 1,452 | 15,320 |
6. DNC | Exclude all Do Not Call | 763 | 14,557 |
7. ELIGIBLES | Qualified Leads | 14,557 |
Review as you usually do.
Are these in line with what you expected?
Compare against previous reports. In case of major changes, investigate.
Instruct AI for Audit Reports
Instruction 03: Audit Reports
Need to create two reports to validate Waterfall rules
1. Criteria QA Report
Take final eligibles from LEADS table. Show following metrics:
- counts by block code, ever 30, current dpd, DNC
- counts by following OTB grouping: Less than 10000, 10000, Greater than 10000
- Compute new variable
OTB_UTIL
(CURR_BAL + 10000
) as % of Credit Limit. Show counts by following grouping:<90%
, exactly90%
,>90%
- count by BScore with this grouping:
<600
,600
,>600
Report Format: Stack the distribution summaries for each specified field vertically. This final table must include a 'Metric'
column that identifies the source of each summary. Show all the categories, even if a category does not have value.
2. Data Quality Report
For the following variables: BScore, Credit Limit, OTB, OTB_UTIL, Current DPD, Ever 30 DPD, limit, curr_bal, cash propensity decile, current util & propensity score
Single table with the following statistics (each a column):
- Zero_Count: Count of explicit 0 values
- Missing_Count: Count of blank/null / NaN
- COUNT / MIN / MAX / Quartiles
Format both as tables, and place them one below the other.
Run ungrouped, or specify custom groupings
Create calculated fields on the fly.
Report format.
Category to be shown even if no value
Create additional custom reports
Get pseudo code, new script
Repeat same process
Follow same process for all subsequent steps
- Confirm understanding with AI
- Review pseudo code
- Copy script into
main.py
and run it
With one difference: we requested for new script
- Copy it below the last one
- Dropdown will now show 2 scripts.
- Select the new script and hit run
xlwings Lite
Dropdown showing two scripts:
- apply_exclusions
- generate_waterfall_report
Previous code
def apply_exclusions(bc
if not leads_df.empty:
except Exception
else:
print("No eligible leads found, so the 'LEADS' ta
print("\n🎉 Script finished successfully!")
New code
import xlwings as xw
from xlwings import script
import pandas as pd
# THIS HELPER FUNCTION IS MANDATORY FOR ALL SCRIPTS ACCESSING
# (Include if not already in your main.py from the previous s
def find_table_in_workbook(book: xw.Book, table_name: str):
"""
Searches all sheets for a table and returns both the she
Returns: (xw.Sheet, xw.Table) or (None, None)
Review Audit Reports
WATERFALL AUDIT | ||
---|---|---|
Metric | Value | Count |
BLOCK_CODE | BLANK | 14,557.00 |
DPD_CURRENT | - | - |
DPD_CURRENT | 1.00 | 9.00 |
DPD_CURRENT | 2.00 | 22.00 |
DPD_CURRENT | 14,526.00 | |
EVER_30P_P12M | 14,557.00 | |
DNC_TAG | 14,557.00 | |
BScore Grouping | <600 | - |
BScore Grouping | 600.00 | - |
BScore Grouping | >600 | 14,557.00 |
OTB Grouping | <10000 | - |
OTB Grouping | 10,000.00 | - |
OTB Grouping | >10000 | 14,557.00 |
OTB Util Grouping | <90% | 14,557.00 |
OTB Util Grouping | 0.90 | - |
OTB Util Grouping | >90% | - |
Have any non-compliant creeped in?
DATA QUALITY REPORT | |||||
---|---|---|---|---|---|
Variable | Non-Missing Count | Missing_Count | Zero_Count | min | |
BEHAVIOUR_SCORE | 14,557.00 | - | - | ||
CREDIT_LIMIT | 14,557.00 | - | - | 15,000 | |
OTB_CURRENT | 14,557.00 | - | - | 11,000 | |
OTB_UTIL | 14,557.00 | - | - | ||
DPD_CURRENT | 31.00 | 14,526.00 | - | ||
EVER_30P_P12M | - | 14,557.00 | - | ||
CURR_BAL | 14,557.00 | - | 4,684.00 | ||
CASH_PROPENSITY_DECILE | 12,362.00 | 2,195.00 | - | ||
UTIL_CURRENT | 14,557.00 | - | 4,684.00 |
Data Quality Report: Pay special attention to zeros and missing values. While this report validates the waterfall output, a full process also includes a preliminary DQ check on the input data before the waterfall begins.
Part 2 : Segment Leads
Objective: Determine outbound calling prioritization
Top Tier to be called first
Tier | Description | Criteria (in Waterfall order) |
---|---|---|
Tier 1 - TOP | Call ASAP | OTB ≥ 50,000 AND Inquiry ≥ 1 in past 15 days |
Tier 2 - MED | Call after Tier 1 | OTB ≥ 25,000 AND (Inquiry ≥ 1 in past 15 days OR Cash Propensity Score Top 5 Deciles) |
Tier 3 - LOW | Call after Tier 2 | All remaining leads |
Many ways to segment - from rules based to ML based, and for multiple purposes. This example shows a rule-based segmentation (based on past analysis) for prioritizing high value outbound calls
Instruct for Segmentation
Segmentation + Profile Report
Instruction 04 : Segment Leads
Categorize leads based on predefined criteria shared below and create lead profile summary report
Use table named LEADS
.
Add a new field called LEADS_CAT, populated with the Tier value Tier 1 - TOP etc. Do this in a waterfall/sequential manner. Create new table called
LEADS_CAT
My deciles range from 1 to 10, with 1 being top or best. Blanks / NaN in decile and propensity score replace with respective medians
Enquiry variable has blanks - replace with zero
Leads Profile Summary
Group by the new leads category tier field and show:
- Counts
- Average balance
- Average credit limit
- Average of util_current
- Average of OTB
- Average Beh.Score
- Average for number of inquiries
- Min, Max, Average of cash propensity decile
- Min, Max, Average of propensity score
Report format: Tiers would be in columns and the metrics above would be in rows. Show all tiers. Put this leads
ALWAYS double check and specify how your deciles are setup
Manage Missing/Blanks. Necessary for stratified sampling in next steps. Ideally do it right at start
Describe how you would like the profile summary report to be setup
Review Lead Segmentation
Part 1 - The Summary
Metric | Tier 1 - TOP | Tier 2 - Call after Tier 1 | Tier 3 |
---|---|---|---|
Count | 1,772.00 | 3,921.00 | |
Avg Balance | 28,942.14 | 33,530.99 | |
Avg Credit Limit | 128,394.47 | 88,684.01 | |
Avg Util Current | 0.20 | 0.36 | |
Avg OTB | 99,452.33 | 55,153.02 | |
Avg Behaviour Score | 705.40 | 673.57 | |
Avg Inquiries (P15D) | 3.94 | 1.22 | |
Min Cash Propensity Decile | 1.00 | 1.00 | |
Max Cash Propensity Decile | 10.00 | 10.00 | |
Avg Cash Propensity Decile | 6.91 | 4.83 | |
Min Propensity Score | 78.00 | 70.00 | |
Max Propensity Score | 876.00 | 912.00 | |
Avg Propensity Score | 376.71 | 543.33 |
- Review summary metrics: Are counts and averages in line with the segmentation criteria?
- Compare against previous profiles: Any major, unexpected shifts from prior runs?
- The summary provides the 'all clear' at 10,000 feet. But the critical work is in the details.
Review Lead Segmentation
Part 2 - The Raw Data Audit
A summary can hide issues. A raw data scan is mandatory. It's the only way to find details like this:-
This 445.5 value is not an error. It's the correct output from a median imputation that returned a float.
CUST_ID | CARD_TYPE | PROPENSITY_SCORE |
---|---|---|
4277937 | GOLD | 501 |
4265647 | GOLD | 224 |
4277490 | PLATINUM | 445.5 |
4279558 | GOLD | 445.5 |
4271142 | SILVER | 607 |
4260167 | SILVER | 117 |
4272034 | SILVER | 843 |
This is an important checkpoint. The next step isn't automatic correction; it's a judgment call based on downstream impact.
In this case, the impact on tertile-based stratification is negligible, so it could be left. For data integrity, my preference is to cast to an integer. I've left it as-is to highlight the importance of running multiple validations and especially raw data scans
Pre-Sampling Checklist
The next two sections - control group setup and test cells setup use stratified sampling. I have consolidated the instructions for demo purposes – in actual practice best to do it one component at a time, validating as you go.
Define your methodology: Methodologies can differ based on use case, business constraints, population size & distributions. Define the methodology you want to follow for your use case - from sampling techniques to the tests you want to run.
Be prepared to iterate: A single instruction might not get the result you want. Review output, assumptions and iterate.
Decide about random seed: the sample splits use
random_state
. If you want to test different splits, you'll need to remove therandom_state
.Execute missing value imputations first. Carry out any missing value imputations before this stage, else AI might use its own judgment (typically median) - which might not be what you want.
Part 3: Setup Control Group
Instruction 04 - Control Group & Reports
Split the final eligible leads into a Test and Control group and run profile validation reports
1: Control Group with Stratified Sampling
- Take the final eligible leads from the
LEADS_CAT
table - Create a stratified random control group of 15%
- Stratified by
CARD_TYPE
& tertiles of - propensity score, current util, beh score, credit limit - If a strata has a single record, assign that to TEST group.
- NO missing value imputation to be carried out. They have been carried out during lead categorization stage.
- In the same Table, add a new field called
GROUP
- 'TEST' or 'CONTROL'.
Specify how you want the sampling done
Specify any custom treatments for data and sampling
2: Test vs. Control Profile Report
Create two separate profile reports, one for numeric and one for categorical. Put them on a new sheet named CONTROL_AUDIT
.
2a Numeric Profile
Variables: limit, util_current, beh score, propensity score
Create a long form vertically stacked single table with the following columns:
- Variable Name
- Metric - Compute MIN / MAX / MEAN / COUNT / STD_DEV and show in separate rows
3 & 4. Test group and Control Values for the metric
5 & 6. P-values: For MEAN row, include:- T-Test p-value (assume equal variance)
- Mann-Whitney U p-value (two-sided)
- Effect Size: For MEAN row, include Cohen's d.
Specify format for your profile and audit reports
Specify statistical tests you want to run
Report 2b: Categorical profile
For CARD_TYPE
Format: Vertically stacked summary table with the following
Review control audit reports
Overall Test vs Control Comparison - Numeric Profile (TEST vs CONTROL) | |||||||
---|---|---|---|---|---|---|---|
Variable | Metric | TEST_Value | CONTROL Value | T-Test p-value | Mann-Whitney U p-value | Cohen's d | |
CREDIT_LIMIT | COUNT | 12,373.00 | 2,184.00 | ||||
CREDIT_LIMIT | MEAN | 88,269.62 | 88,461.54 | 0.89 | 0.97 | -0.00 | |
CREDIT_LIMIT | STD | 61,618.57 | 61,913.42 | ||||
CREDIT_LIMIT | MIN | 15,000.00 | 15,000.00 | ||||
CREDIT_LIMIT | MAX | 300,000.00 | 300,000.00 | ||||
UTIL_CURRENT | COUNT | 12,373.00 | 2,184.00 | ||||
UTIL_CURRENT | MEAN | 0.28 | 0.28 | 0.94 | 0.97 | 0.00 | |
UTIL_CURRENT | STD | 0.21 | 0.21 |
Variable | Category | TEST | CONTROL | TEST % | CONTROL % | |
---|---|---|---|---|---|---|
CARD_TYPE | GOLD | 5,028.00 | 889.00 | 40.64 | 40.71 | |
CARD_TYPE | PLATINUM | 2,143.00 | 379.00 | 17.32 | 17.35 | |
CARD_TYPE | SILVER | 5,202.00 | 916.00 | 42.04 | 41.94 |
Chi-Square Statistic: | 0.008 |
P-Value: | 0.996 |
Cramer's V (Effect Size): | 0.001 |
Evaluate profile and statistical test values for consistency of splits. Tweak parameters and iterate.
Always check AI understanding & pseudo-code. Have AI walk you through code if you want to dig deeper.
Part 4: Setup Test Cells for Offers
Instruction 06: Create Test Cells for Offers
Part 1: Test Cell Assignment
- Take the
LEADS_CAT
TABLE and ONLY the TEST group - I have two offers to test -
OFFER_9.99
/OFFER_10.99
- Create stratified cut of 50% each.
- Stratify by
CARD_TYPE
, & tertiles of - propensity score, current util, beh score, credit limit - NO missing value imputation to be carried out. They have been carried out during lead categorization stage.
- If a strata has a single record, assign that to
OFFER_10.99
group. - Add a new column 'OFFER'. Update existing
LEADS_CAT
table only.
Part 2: Test Cell Profile Report
- In a new sheet titled
OFFER_PROFILE
- Create exactly the same reports that you did for the test and control group - instead of test and control this time it would be these 2 offer groups now.
- In addition, create the same set of reports for the 3 Tiers and put them below the overall report from the previous step.
- Share your understanding of the requirement as well as the pseudo code.
- Create a new, runnable script. Name the script
E_TEST_CELLS
. - Follow AI Coder Instructions carefully - this is an even trickier piece.
Specify details of how you want the test cells to be set up
Profile and audit report formats can remain same as in control group audit report
Review test cell profile reports
Overall Offer Comparison - Numeric Profile (OFFER_10.99 vs OFFER_9.99) | OFFER 10.99_Value | OFFER 9.99_Value | T-Test p-value | Mann-Whitney U p-value | Cohen's d | |
---|---|---|---|---|---|---|
Variable | Metric | |||||
CREDIT_LIMIT | COUNT | 6,187.00 | 6,186.00 | |||
CREDIT_LIMIT | MEAN | 88,045.90 | 88,493.37 | 0.69 | 0.96 | -0.01 |
CREDIT_LIMIT | STD | 61,135.16 | 62,102.42 | |||
CREDIT_LIMIT | MIN | 15,000.00 | 15,000.00 | |||
CREDIT_LIMIT | MAX | 300,000.00 | 300,000.00 | |||
UTIL_CURRENT | COUNT | 6,187.00 | 6,186.00 | |||
UTIL_CURRENT | MEAN | 0.28 | 0.28 | 0.91 | 0.95 | -0.00 |
UTIL_CURRENT | STD | 0.21 | 0.21 |
Variable | Category | OFFER_10.99 | OFFER_9.99 | OFFER_1 | OFFER_9.99, |
---|---|---|---|---|---|
CARD_TYPE | GOLD | 2,512.00 | 2,516.00 | 40.60 | 40.67 |
CARD_TYPE | PLATINUM | 1,073.00 | 1,070.00 | 17.34 | 17.30 |
CARD_TYPE | SILVER | 2,602.00 | 2,600.00 | 42.06 | 42.03 |
Chi-Square Statistic: | 0.008 |
P-Value: | 0.996 |
Cramer's V (Effect Size): | 0.001 |
Evaluate, assess and iterate
Final Campaign Setup & Measurement Plan
Total Population | 25,000 |
---|---|
Eligibles After Waterfall | 14,557 |
GROUP | OFFER | Tier 1 | Tier 2 | Tier 3 | Total |
---|---|---|---|---|---|
TEST | 10.99 | 769 | 1,661 | 3,757 | 6,187 |
9.99 | 745 | 1,670 | 3,771 | 6,186 | |
Total | 1,514 | 3,331 | 7,528 | 12,373 | |
CONTROL | 2,184 | ||||
Grand Total | 14,557 |
Primary Measurement Plan
- Overall Lift: Test vs. Control performance.
- Segment Performance: Tier 1 vs. Tier 2 vs. Tier 3 analysis.
- Offer Performance: 10.99% vs. 9.99% analysis.
Secondary Analysis & Finer Cuts
- Offer vs. Tier / Card Type / Tier Level Lifts / sub-segments
- Depending on response rate, smaller cuts could lead to insufficient sample size for sub-segments
- If direct comparison not statistically valid - use regression models to isolate variable impact
Setup campaign with measurement in mind
Campaigns: Mandatory Rules
Never release a list unless validated. Ever.
You are the Pilot – the flight doesn’t take off till you are confident. No matter the pressure. It is not worth it.
- Initial runs are almost always wrong. Dig. Iterate. Rerun.
- Audit reports aren't enough. use filters, pivots, direct queries
- Automated codes + wrong parameters = Garbage. Double-check params before every run.
- Automated codes + missed errors = Land mine. Check console logs.
- Blanks and Missing create havoc – identify and treat
- Inverted decile = Disaster. Verify the range (1-10 or 0-9) and which end is ‘top’. Double-check, even if you built the model.
- Beware of row shifts. Cust A's name against Cust B's info
- Final CSV - Open in Notepad++ first. Look for truncation & file format errors.
- Campaign workbook is a permanent audit trail. Must contain everything: exclusions, segments, audits, approvals.
- Business constraints can override statistical purity. The goal is a robust, practical solution, not a perfect one.
- Something feels off? It probably is. Trust your gut. Dig.
From Workflows to Standalone Tools
The apps below are examples of standalone xlwings Lite tools built using exactly the same protocols. These are not tutorials, but fully functioning, open-source applications that demonstrate advanced use cases like database connections, web scraping and API calls.
Live app + full code on my site to use as a reference for your own projects.
Explore Live Apps
Web Scraper
Transform Excel into a powerful web scraping platform using Jina AI and Google Gemini
- Uses Jina AI API for webpage rendering to markdown
- Google Gemini analyzes content for intelligent data extraction
- Process batches of URLs with custom column specifications
Technical Analysis
Create professional technical analysis pattern recognition
- Pulls live data from Yahoo Finance via custom API
- Gemini Vision API analyzes charts for technical patterns
- Generates PDF/HTML reports with embedded charts
Database & ML
Connect to any database with custom FastAPI layer for data analysis and machine learning
- Connects to PostgreSQL/MySQL via web API layer
- Pull random records; run custom queries; explore data
AI Workflows
Automate Excel workflows with AI scheduler Flash or GPT-4o
- Automatic schema detection with structured data
- Identify categorical and numerical variables in datasets
https://app.tigzig.com/xlwings-starter
TIGZIG: Micro-Apps for Analytics
25+ apps: Database AI / xlwings Lite / Automation / Quants
![Open Source]
Get the full module and workbooks:
[app.tigzig.com])(undefined)
Path: xlwings Lite: Practice Lab
https://app.tigzig.com/landing
TIGZIG: Co-Analyst ![OSS] [More Apps]
Free. No Sign-Up Required.
What do you want to do today?Search tools...
FEATURED
xlwings Lite: Practice Lab [NEW] ⭐
Learn xlwings Lite with Practical Examples
A Hands-on guide to get up and running fast with xlwings Lite
- AI-ready instructions & examples
- Step-by-step learning guides
- Interactive workbooks
Amar Harolikar
Specialist - Decision Sciences & Applied Gen AI
Builder of app.tigzig.com