COFES
The Congress on the Future of Engineering Software Register | Login
 

COFES 2011
April 14-17, 2011
Scottsdale, Arizona
The Scottsdale Plaza Resort

  Search
COFES Blog
Aug 15

Written by: Brad Holtz
8/15/2009 

This is a data reshaper for Excel for data that does will need to be moved into two columns: one for the column name and one for the data. It still could use an error rountine.  Be sure to read the notes and assumptions.  Comments and improvements are welcome.

Qualifier - This IS for data that does not reflect the column.

It takes Data like this:

ID Truck Car Bike
1234 $4,700    
1235 $32,000 $12,000 N/A
1236 N/A Don't Know $300

 and coverts it into Data like this:

ID Vehicle Price
1234 Truck $4,700
1235 Truck $32,000
1236 Truck N/A
1234 Car  
1235 Car $12,000
1236 Car Don't Know
1234 Bike  
1235 Bike N/A
1236 Bike $300


' ColumnReshapeType2 Macro
' Reshape Column Data
'
'creative commons 2009 (cc) Cyon Research Corporation
'
'This macro will take data that has multiple columns related to a single question,
'and reshape them into a single column, along with the record identifier.
'
'NOTE: BEFORE YOU BEGIN
' 1. Insert a TWO blank columns after Column A
' 2. Give the columns a title in the header
'    Column "B" will hold the information about which column the data came from
'    Column "C" will hold the value of the data
' 3. There is no undo - make sure you have a backup of the worksheet
'
'Assumptions:
'  1. Record identifier is located in Column "A"
'  2. An uncertian number rows of headers above data
'        AND
'        Column header to be used as data for Column "B" is in row 1
'  3. Data in the cell for the column DOES NOT reflect the column header and
'        column header needs to be copied to Column "B"
'           (for instance,
'            if the question is "What is the cost of your vehicle" and the
'            choices are shown in mulitple columns as numbers or text,
'            where the Column "F" is "Truck" and Column "G" is "Car"
'            and data in Column "F" and Column "G"  looks like "$4500" or "N/A")
'
' Keyboard Shortcut: Ctrl+r
'
    'Get Range to Copy
       DataShaperForm.Show 'Show Range Input form
   
    'Define form data
       'Data in RowXxx is the number of the rows that contain data
       RowStart = DataShaperForm.RowStart.Value 'Starting Range Cell for Row Calculation
       RowEnd = DataShaperForm.RowEnd.Value 'Ending Range Cell for Row Calculation
    
       'data in ColumnXxx is the letter of columns that need to be reshaped (excluding the ID column A)
       ColumnStart = DataShaperForm.ColumnStart.Value 'Starting Range Cell for Column Calculation
       ColumnEnd = DataShaperForm.ColumnEnd.Value 'Ending Range Cell for Column Calculation
      
       ColumnName = "D1" 'The name of the column is in the header in row 1 in the first column
      
   
    'NEW VARIABLES
       'data for columns to be copied
       FirstDataCell = ColumnStart & RowStart
       LastDataCellInColumn = ColumnStart & RowEnd
       CellOfLastDataColumn = ColumnEnd & RowStart
   
       FirstIDCell = "A" & RowStart
       LastIDCell = "A" & RowEnd
   
       'Loop counts
       LoopCount = 1
       NumOfRowsToCopy = RowEnd - RowStart + 1
       increment = NumOfRowsToCopy + 1
      
       'Get ColumnCount
       ColumnRange = FirstDataCell & ":" & CellOfLastDataColumn
       NumOfColumnsToCopy = Range(ColumnRange).Columns.Count + 0
         
    'Loop
       For LoopCount = 1 To NumOfColumnsToCopy 'For each column to be copied...
      
       'Reset Ranges to be Copied
       idrange = FirstIDCell & ":" & "A" & RowEnd
       DataRange = FirstDataCell & ":" & LastDataCellInColumn
              
       'Copy ID and Data from First of the columns to be shaped to the new location
       Range(idrange).Copy Range(idrange).Offset(increment, 0)
       Range(ColumnName).Copy Range(idrange).Offset(increment, 1)
       Range(DataRange).Copy Range(idrange).Offset(increment, 2)
      
       'Reset the increment to the new end of column
       increment = increment + NumOfRowsToCopy + 0
      
       'Delete the Column just reshaped
       DeleteThisRange = ColumnStart & "1:" & ColumnStart & RowEnd
       Range(DeleteThisRange).Delete
      
       Next LoopCount

 


 

Creative Commons License
Excel Data Reshaper by Cyon Research Corporation is licensed under a Creative Commons Attribution-Share Alike 3.0 United States License.
Requests for permissions beyond the scope of this license may be addressed via email to info@cyonresearch.com.

Tags:

9 comment(s) so far...

Re: Second Data Reshaper for Microsoft Excel

Cool! I am always amazed at the things possible in excel. However, I often point out how easy data based things can be in a database. For example If you had the same data in Microsoft Access you could just do a self join on the table and return the same results.

I created a small three column table named tbl_input with columns to match your input columns (ID,Truck,Car,Bike) I input data to match. Then made the following sql query to return the same results

(SELECT tbl_input.ID, "Truck" AS Vehicle, tbl_input_1.Truck
FROM tbl_input AS tbl_input_1 INNER JOIN tbl_input ON tbl_input_1.ID = tbl_input.ID)
UNION ALL
(
SELECT tbl_input.ID, "Car" AS Vehicle, tbl_input_1.Car
FROM tbl_input AS tbl_input_1 INNER JOIN tbl_input ON tbl_input_1.ID = tbl_input.ID)
UNION ALL
(
SELECT tbl_input.ID, "Bike" AS Vehicle, tbl_input_1.Bike
FROM tbl_input AS tbl_input_1 INNER JOIN tbl_input ON tbl_input_1.ID = tbl_input.ID);

ID Vehicle Truck
1234 Truck $4,700
1235 Truck $32,000
1236 Truck N/A
1234 Car
1235 Car $12,000
1236 Car Don’t Know
1234 Bike
1235 Bike N/A
1236 Bike $300

I had more time in inputing the data than writing the query

mike J.

By Michael Jackson on   8/15/2009

Re: Second Data Reshaper for Microsoft Excel

Mike,
Yes it was much easier to do this in MS Access. However, a lot --more than I would care to quote-- of people I work with are intimidated by Database applications MS Access included. Both Brad and I are MS Access junkies, but we realize that not everyone is. Also some people don’t like switching back and forth between applications; they’d prefer to do it all in one application and most calculations verses data query are best in EXCEL. I’ve several dozen MS EXCEL I’ve posted on the Microsoft Office templates online site and about ½ a dozen MS Access applications that will be posted soon, just prior to Office 2010 (Agreement I have with the MS Access Dev Team).

By Brian Seitz on   8/15/2009

Re: Second Data Reshaper for Microsoft Excel

Mike,

There are at least two reason much of this data is kept in Excel. For survey data, there are MANY different question types, each of which requires a different table structure. In our most recent survey, we had at least five types of questions. Determining an optimal structure for such data is not trivial and if you aren't careful, you can end up with misleading results. Second, data cleaning, inspection, and manipulation is much more visual, and very straight forward. It also lends itself to rapid experimentation and testing. Yes, you can do the same with databases (You've seen mine - you know I can), but you are at least one step remote from the data.

By Brad Holtz on   8/16/2009

Re: Second Data Reshaper for Microsoft Excel

One thing I forgot to mention: Your example is explicit. While simple for a three-column dataset, it becomes problematic for my dataset, which has well over 500 columns of data for each respondent.

By Brad Holtz on   8/16/2009

Re: Second Data Reshaper for Microsoft Excel

One thing I forgot to mention: Your example is explicit. While simple for a three-column dataset, it becomes problematic for my dataset, which has well over 500 columns of data for each respondent.

By Brad Holtz on   8/16/2009

Re: Second Data Reshaper for Microsoft Excel

You are right. I understand that there are many people that feel more comfortable with excel and your application may be completely justified. When you get large amounts of data it can become unmanageable in a spreadsheet. I just like to point out the flexibility of data in a database. People often see a database as this rigid thing.

To give a more generic example. If you had structured the data differently on input which would probably been a better choice anyway. You could have structure it like

UserID
Question
Answer

Then the base data would look something like what you reshaped to

surveyQs
ID User question answer
1 1234 truck $4,700
2 1235 truck $32,000
3 1236 truck N/a
4 1234 car
5 1235 car $12,000
6 1236 car Don't Know
7 1234 bike
8 1235 bike N/A
9 1236 bike $300

Then you could do a crosstab query to get to the other format you were using

TRANSFORM First(surveyqs.answer) AS FirstOfanswer
SELECT surveyqs.id
FROM surveyqs
GROUP BY surveyqs.id
PIVOT surveyqs.qtype;

User bike car truck
1234 $4,700
1235 N/A $12,000 $32,000
1236 $300 Don't Know N/a

Once you get the data in a database then all sorts of things are possible. One of those things would be to query it from Excel.

"To the man who only has Excel, Everything looks like a spreadsheet"

By Michael Jackson on   8/17/2009

Re: Second Data Reshaper for Microsoft Excel

You are right. I understand that there are many people that feel more comfortable with excel and your application may be completely justified. When you get large amounts of data it can become unmanageable in a spreadsheet. I just like to point out the flexibility of data in a database. People often see a database as this rigid thing.

To give a more generic example. If you had structured the data differently on input which would probably been a better choice anyway. You could have structure it like

UserID
Question
Answer

Then the base data would look something like what you reshaped to

surveyQs
ID User question answer
1 1234 truck $4,700
2 1235 truck $32,000
3 1236 truck N/a
4 1234 car
5 1235 car $12,000
6 1236 car Don't Know
7 1234 bike
8 1235 bike N/A
9 1236 bike $300

Then you could do a crosstab query to get to the other format you were using

TRANSFORM First(surveyqs.answer) AS FirstOfanswer
SELECT surveyqs.id
FROM surveyqs
GROUP BY surveyqs.id
PIVOT surveyqs.qtype;

User bike car truck
1234 $4,700
1235 N/A $12,000 $32,000
1236 $300 Don't Know N/a

Once you get the data in a database then all sorts of things are possible. One of those things would be to query it from Excel.

"To the man who only has Excel, Everything looks like a spreadsheet"

By Michael Jackson on   8/17/2009

Re: Second Data Reshaper for Microsoft Excel

Mike,

If life were only so simple!

I just spent the last several weeks reshaping, testing and evaluating the data. In the end, 30 questions ended up with 9 different data structures in 18 tables. Some of them needed to be available in different shapes simultaneously in order to accomplish specific goals. It seems strange that such a seemingly simple issue becomes so complex, but much of that complexity lies in what I want to do with the data. On the other hand, what I can do with the data is pretty useful.

Brad

By Brad Holtz on   8/17/2009

Re: Second Data Reshaper for Microsoft Excel

Thanks for giving good information.I really like it.Thanks to you.mcse practice exams

By mcse practice exams on   1/14/2010

Your name:
Your email:
(Optional) Email used only to show Gravatar.
Your website:
Title:
Comment:
Add Comment   Cancel 
DISCLOSURE: The US Federal Trade Commission mandated in December of 2009 that bloggers must disclose any material connection and compensation received for blog posts to inform consumers of paid endorsements. The blog published here is completely my own and Cyon Research receives no compensation for its content. However, readers should assume that Cyon Research currently has, has had in the past and is likely to seek a business relationship with any company mentioned here. Likewise, Cyon Research employees may not directly own shares in any company reported on here. However, it is likely that mutual funds or other investment vehicles contain shares that are not under the direct control of company employees.
COFES Blog Search
©Copyright 2010 Cyon Research Corporation Terms Of Use | Privacy Policy
Site Credits