0001 1 2 3 4 THE COUNCIL FOR 5 RESEARCH EXCELLENCE 6 7 8 9 AUGUST 8, 2007 10 11 12 Omni Hotel 13 21 East 52nd Street 14 New York, New York 15 12:20 p.m. to 4:30 p.m. 16 17 18 19 20 21 REGENCY REPORTING, INC. 22 Certified Shorthand Reporters & Videographers 23 425 Eagle Rock Avenue 575 Madison Avenue 24 Roseland, NJ 07068 New York, NY 10022 25 www.regencyreporting.net 1-866-268-7866 0002 1 A P P E A R A N C E S: 2 3 BOARD MEMBERS: 4 MARK KALINE, Chairperson 5 SHARI ANNE BRILL 6 TIM BROOKS 7 JOANNE BURNS 8 MICHELE BUSLIK 9 SUSAN CUCCINELLO 10 PAUL DONATO 11 COLLEEN FAHEY RUSH 12 NANCY GALLAGHER 13 DON GLOECKLER 14 JEAN GOLDBERG 15 DAVID POLTRACK 16 GEORGE IVIE 17 RICK KEILTY 18 PAT LIGUORI 19 JESSICA PANTANINI 20 CERIL SHAGRIN 21 HOWARD SHIMMEL 22 STEVE STERNBERG 23 24 25 0003 1 A P P E A R A N C E S (Continued): 2 3 KATE SIRKIN (via telephone) 4 VICKI CHAMPLIN (via telephone) 5 IRA SUSSMAN 6 JACK WAKSHLAG 7 RICHARD ZACKON 8 REBECCA MITCHELL 9 10 11 PRESENTERS: 12 MIKE BLOXHAM 13 JIM SPAETH 14 BILL MOULT 15 MICHAEL HOLMES 16 17 18 19 20 21 22 23 24 25 0004 1 MR. ZACKON: Unfortunately, I 2 want to acknowledge Rebecca for the 3 toys. I question the rocks. 4 Welcome, everyone. Thanks for 5 getting here on a hot day, a muggy day, 6 travel restricted day and still get here 7 for a long day. I think the agenda -- 8 A SPEAKER: Kate Sirkin has 9 joined you. 10 MR. ZACKON: Welcome. 11 MS. SIRKIN: Hi. 12 MR. ZACKON: You missed nothing. 13 It is tough getting around the city 14 today. 15 MS. SIRKIN: I'm glad I'm not 16 there. 17 MR. ZACKON: How about I turn it 18 over to Mark. 19 MR. KALINE: I think we're going 20 to have people joining throughout the 21 process this afternoon as they arrive 22 via whatever mode of transportation or 23 via phone. I just want to thank 24 everybody for making the effort to get 25 here. I think it is remarkable that any 0005 1 meeting gets started within a half hour 2 within the original start time the way 3 things are going within the city today. 4 We do have a full agenda. There 5 would be a lot of healthy discussion 6 around some of the issues and I look 7 forward to that. So with that, I think 8 I'll just try to cut it short and let's 9 get to some of the committee reports. 10 MR. ZACKON: We did a survey and 11 I thank Colleen for doing the bulk of 12 the work on that. There was a sense of 13 reduce the amount of time for the 14 committee report at the meeting. No one 15 thought we were spending too little 16 time. 17 We have one big issue. Media 18 consumption and engagement. The other 19 ones will go quickly. There are reports 20 in here which I'll talk about in a bit 21 and the bulk of what today is about is 22 not just a business meeting, but sit 23 back and think meeting where we look at 24 how we're doing, what we should be doing 25 differently or maybe not doing, 0006 1 et cetera. 2 So our first committee is the 3 steering committee. And Mike is not 4 here. I'm going to tell you this is how 5 quickly we will go. If anyone wants 6 more information, if you look in your 7 packet -- if you look in your packet, 8 I'm going to wait just so you know for a 9 couple minutes before we take roll 10 because people are still coming in. 11 I believe we have Colleen and 12 Kate on the phone, is that right? 13 MS. RUSH: Colleen is here. 14 MS. SIRKIN: Kate is here. 15 MR. ZACKON: If we have more 16 phone people we will ask. It was Tim 17 Brooks' suggestion that maybe we have 18 something like a briefing for these 19 meetings. This was my first cut attempt 20 at a briefing book. There are five 21 pages, one for each committee. Those 22 are the issues as best as I know where 23 they stand. I do not speak for the 24 committee chairs on any of these. 25 We can go forward as a practice 0007 1 and have them reviewed and updated by 2 the committee chair to provide them to 3 you in a more timely basis. So, by way 4 of the briefing book, I can tell you the 5 issues on the steering committee and at 6 the last meeting we voted through the 7 bylaws, so that's complete for the 8 steering committee. 9 The issues are still membership 10 and what new areas of research the 11 council will take on, and those are 12 issues we will speak to today and will 13 be followed up by the steering 14 committee. 15 If there is someone from the 16 steering committee that would like to 17 add something to those, members Susan, 18 Henry, Don, Pat, Ceril, Ira and Jack. 19 By the way, I noticed in our bylaws that 20 we require 11 people on the steering 21 committee; so, it would appear that some 22 people can step up and participate on 23 the committee. 24 Howard, I think you're a member 25 of that committee and I don't know if 0008 1 you're listed. So that's the steering 2 committee. Anything else to be said 3 about the steering committee? 4 You've done a good job. Very 5 good. Thank you. 6 Next committee is our 7 Nonresponse, Nonresponse Bias Committee, 8 and that's the committee that Ceril is 9 chairing. If you look on the second 10 page that lays it all out. Ceril, I'll 11 ask you between -- we'll talk after the 12 meeting, if we could get something that 13 addresses these meetings and updated. 14 MS. SHAGRIN: No problem. 15 MR. ZACKON: But the floor is 16 yours. 17 MS. SHAGRIN: Well, in terms of 18 this committee, it is -- the project has 19 been closely monitored. I actually get 20 a detailed project status report every 21 two weeks and that tells me exactly 22 what's going on and what's working and 23 what isn't working and if there are any 24 problems. We had a follow up meeting 25 with the professors to make sure we're 0009 1 all on the same track in terms of the 2 analysis. And we'll have another 3 meeting to discuss in more detail some 4 of the analysis that the professors are 5 going to do. 6 And so let me share with you some 7 really good news, which is that we are 8 exceeding the expectations in terms of 9 the response rates that we thought we 10 would get. And when Paul Labrochus 11 wrote the proposal and he estimated what 12 kind of response rates we would get, he 13 estimated on the cooperating basic 14 metered households we would get 15 57 percent response rate from the 16 mailings and 79 percent with the 17 in-person recruitment. 18 To date, with just first and 19 second metered mailings, we have an 20 84 percent response rate, which means 21 we'll do fewer in-person response rates. 22 MR. ZACKON: That's thunderous 23 applause. 24 MS. SHAGRIN: On the reviewing 25 basic which I find even, you know, we 0010 1 estimated 43 percent response rates and 2 right now, to date, the actual is at 3 50 percent, and we expect these response 4 rates to increase because the second 5 mailing has just been out for a short 6 period of time. This is just on the 7 first group of households. If it 8 continues that way, as we do more 9 turnover households, I'll be really 10 happy. 11 And the goal for the diary 12 dispense rates was 80 percent. Response 13 rate in total would be in-person and, to 14 date, we are at 69 percent which, again, 15 I think is very positive, given the 16 status that we're at. 17 So the good news is it is very 18 closely monitored by Nielsen, as well as 19 by the committee, and so far things are 20 working well. 21 MR. ZACKON: Ceril, I apologize, 22 it is not in your kits. I thought it 23 would be included. We do not get a 24 bi-weekly report. 25 MS. SHAGRIN: Right. We were 0011 1 going to share the last one and I didn't 2 bring it. 3 MR. ZACKON: I'll see if Rebecca 4 can get that. 5 MS. SHAGRIN: I can e-mail it if 6 people are interested. 7 MR. ZACKON: So it is there. 8 Good job. 9 MS. SHAGRIN: I didn't see it as 10 I was looking through because it wasn't 11 attached. 12 MR. ZACKON: Right. 13 MS. SHAGRIN: If you look at that 14 at your leisure, you can see how 15 detailed that report is and I get that 16 every two weeks. So if there is a red 17 flag, we can meet or we can address it 18 and we can take care of it. We are 19 planning on having the process audited 20 by Ernst & Young and we still have to 21 work out the details of that audit. 22 MR. ZACKON: In terms of timing? 23 MS. SHAGRIN: We're on schedule. 24 MR. ZACKON: In terms of budget? 25 MS. SHAGRIN: We're on schedule. 0012 1 MR. ZACKON: Congratulations. 2 MR. GLOECKLER: Better response 3 rates might come in under budget. 4 MS. SHAGRIN: We might come in 5 under budget. That was the last piece 6 of the budget that Nielsen threw at us. 7 Because the original proposal was in 8 person and now we're using RTI. So the 9 fewer RTI calls that have to be made, 10 the less the cost will be. 11 MR. IVIE: Of course, you just 12 gave us the whammy. 13 MS. SHAGRIN: I know that. 14 That's why I didn't say that. Too soon 15 to tell. I won't commit to it. I am 16 glad they're less than we expected; 17 therefore, the cost will go up. 18 MR. ZACKON: We had coals ready 19 to rake you over in that scenario. 20 MS. SHAGRIN: I was prepared to 21 go out and do the final stuff myself. 22 MR. ZACKON: That's good news, 23 indeed. 24 Just a question. Is there a 25 snowball's chance in New York City today 0013 1 that we'll have something to show at the 2 client meeting, Nielsen's client meeting 3 in February from this study? 4 MS. SHAGRIN: I believe we will 5 have some initial findings for that. 6 That's part of it. We'll certainly be 7 able to talk about differential response 8 rates. 9 MR. BROOKS: Could I ask for more 10 on the time line? Obviously, there's 11 nothing on the report here about it. 12 MS. SHAGRIN: I will be happy to 13 update the time line and give it to you. 14 I did not bring the detailed time line 15 with me. I know they're scheduled to 16 start the in-person call in September. 17 MR. BROOKS: Is there a field 18 period for that? 19 MS. SHAGRIN: There is a field 20 period for that. Part of it is we did 21 the turnover homes that we have, so now 22 there is more homes that we have to do. 23 So there's the first group of homes and 24 the second group of homes. 25 I do have the detailed time line. 0014 1 I will verify with Nielsen that there 2 are no changes to that time line and 3 then I will e-mail it so you can all get 4 a copy. 5 MR. BROOKS: What is the target 6 for the completion of the whole thing? 7 MS. SHAGRIN: I'm sorry, but I 8 don't remember the exact date, Howard. 9 I think it is the end of 2008 before we 10 have everything. 11 MR. SHIMMEL: Last mailing, the 12 last group of sample that we're using 13 accepting basics is going to be mailed 14 in the beginning of October. It is 15 going to be people who exited the sample 16 in September. 17 MS. SHAGRIN: Right. 18 MR. SHIMMEL: It is going through 19 the first process, first mailing, second 20 mailing, sending RTI. 21 MS. SHAGRIN: All the analysis 22 done by the end of 2008. 23 A SPEAKER: Vicky Champlin is 24 joining your conference. 25 MR. ZACKON: Welcome, Vicky 0015 1 Champlin. 2 A SPEAKER: Actually, it is 3 Karen. I'm calling for Vicky Champlin. 4 She will join you in just a few moments. 5 MR. IVIE: I think a key step in 6 the process is the data analysis plan, 7 if I remember correctly, you guys are 8 still working on. 9 MS. SHAGRIN: What we're doing 10 with data, there was a meeting held. 11 MR. ZACKON: I apologize to the 12 people on the air. We can't do business 13 with commercials on the air. Thank you, 14 even though we're in that business. 15 MS. SHAGRIN: We did have a 16 session with Nielsen and the professors 17 and I attended that session to talk 18 about the analysis that was going to be 19 done, how they were going to analyze the 20 data, what data was going to be shared, 21 and then there was a follow-up meeting 22 between the professors and Nielsen in 23 terms of determining the criteria for 24 analysis. 25 The next step is supposed to be 0016 1 Nielsen presenting to the professors or 2 to the academics exactly what they plan 3 to analyze. And then after that, we 4 will get from Bob Groves and his group 5 what they plan to analyze. And as soon 6 as that first step is done, then we want 7 to get the whole committee involved in 8 reviewing the plans for analysis. 9 MR. ZACKON: We're going to come 10 back to the meeting consumption and Ira 11 before he goes. I have some mea culpas 12 to make to the room. This is Richard. 13 First, we had a conversation at 14 the last meeting about an annual report, 15 which I only remember was promised when 16 I read through the transcripts in 17 preparation for this meeting. So I 18 don't have it for this meeting, and I 19 will get that out by the end of the 20 month. That is an annual report for the 21 first two years. I'll modify it to make 22 it a calendar year. 23 Second thing, there are two RFPs. 24 I'm a bottleneck if you want to know 25 what it looks like, between Nielsen's 0017 1 lawyer and Nielsen's communication 2 department. I'm going to end that 3 bottleneck and Ira's RFP will 4 accelerate. I promise all my best 5 efforts to have them out. 6 The last piece, and this is not 7 so much an apology. People requested 8 minutes of the meetings which I think is 9 a smart idea, even though we have full 10 transcripts. We will have minutes of 11 this meeting out. I'll take the 12 transcripts and have minutes prepared 13 going back for at least two or three 14 meetings. We'll have that as part of 15 the formal report and part of the 16 report. 17 I apologize. We will do better. 18 And to show how much better, we'll 19 showcase Ira Sussman's marketplace 20 issues. It started off to try to 21 understand marketplace issues, to see if 22 we could do some research to help 23 improve some of the conditions that we 24 have been seeing, especially on the 25 local level. We had somebody do a 0018 1 professional interviews, focusing on 2 local TV and viewing professionals to 3 see the top issues, to identify any and 4 actually act upon any of those. 5 Most of the topics that came back 6 were around problems with diaries. 7 Since that, Nielsen has announced 8 electron measurement in the market, so 9 it kind of changed or the focus or 10 importance of that, should we spend 11 money on some diary issues that are 12 supposedly going away. It may relieve 13 some issues, it may create some new 14 ones. 15 We were thinking about this group 16 going on hiatus, and rather than doing 17 that, we posted the results of those 18 interviews on the website so everybody 19 has access to see what we achieved from 20 that, from our committee. But as the 21 next step, we decided to put an RFP out 22 requesting proposals on projects which 23 can go ahead, cross the gamut in the 24 RPF. We're identifying a number of 25 topics that are of interest. But 0019 1 letting suppliers come to us with what 2 would they like to research, that might 3 help with practices, as opposed to the 4 group coming up with great ideas. 5 We're putting it out to the 6 industry and put together a research 7 proposal and maybe, hopefully, by the 8 end of the summer or early fall, look 9 through a number of those and see if 10 there is anything that merits spending 11 some of this R&D money. 12 That's where we are today. We 13 have a draft of the RFP in the kit for 14 review. Once we get approval on that, 15 we will send it out and move forward on 16 evaluating some projects that might be 17 of value early next year. 18 Any questions? I'm not spending 19 money. Nobody is asking questions. 20 MR. BROOKS: Congratulations to 21 the committee. I believe you completed 22 the study and had it posted. Would 23 there be some value, perhaps, in being 24 more proactive in releasing this data to 25 the first wave, in your opinion, to the 0020 1 larger community that we represent out 2 there? 3 MR. SUSSMAN: It is available to 4 anybody that goes on the website. As 5 far as if we want to do it through a 6 press release, probably we want to take 7 a look at it and see if it is in the 8 right format. It may not be the way we 9 would like to send something out, but if 10 everybody thinks that's valuable, we can 11 look at how do we package that to get 12 some play. 13 MR. BROOKS: Transparency was a 14 value of this conversation. I think its 15 operations are pretty dense or opaque to 16 most Nielsen clients and those who 17 aren't represented in this room. 18 Anything we can do to show the Nielsen 19 client base, the buyers and sellers, 20 what this group is accomplishing, I 21 think would be good. Obviously, it 22 should be better. Posting on the 23 website is good for a step and I think 24 it would be more proactive. 25 MR. ZACKON: You raised a couple 0021 1 valid points. Should there be a 2 publicity committee which decides what 3 gets put out? 4 MS. BUSLIK: No. 5 MR. BROOKS: Personally, I'm not 6 sure that press releases and trying to 7 drum up press is necessarily the right 8 way to go. But communications to a 9 broader base of Nielsen clients, buyers 10 and sellers, I think, is, and some 11 professionals who can help with that. 12 MR. ZACKON: That is what I 13 meant. 14 MS. BUSLIK: They do that. It is 15 always written up and A2/M2 initiative 16 updates. It is written up about what 17 the council has done. I'm somewhat 18 concerned about press releases because 19 they always get misinterpreted by 20 certain people, and I think then we're 21 at the point of answering, that becomes 22 a job for us, no, you didn't determine 23 this correctly. 24 But as far as I know, Nielsen is 25 always writing about the different 0022 1 things that are happening on the 2 council. 3 MR. ZACKON: Steve. 4 MR. STERNBERG: As far as I know, 5 nobody I know in the industry than us 6 has any idea what the council is doing. 7 I hear all the time is that council 8 still in existence? Are you doing 9 anything? Very few people put out what 10 the Nielsen puts out, because they don't 11 really say anything. 12 If we have one or two people in 13 charge of doing that like a corporate 14 communication type of thing, we would be 15 in charge of handling the type of press 16 that we get and, you know, we could 17 certainly gear it to what we want the 18 press to cover. 19 MR. LIGUORI: Would it help 20 instead of burying our stuff in one of 21 their longer multi-press releases, that 22 we request that they put it out 23 separately so at least we're in the 24 headline as opposed to page 3? 25 MR. ZACKON: Let me suggest if 0023 1 I'm going to do an annual report, which 2 I said by the end of August, wouldn't 3 that be something Nielsen could take and 4 appropriately communicate with and 5 anyone on the council could feel free to 6 do that. Maybe it should be geared for 7 that purpose. 8 MR. SHIMMEL: Richard, you and I 9 should have a conversation with Jack 10 Loftus or Karen Watson about creating a 11 communication partner for the council. 12 MR. BROOKS: I believe there are 13 quarterly updates. Monthly, pardon me, 14 which are distributed to the whole 15 client base. I don't know monthly, but 16 I would say more than once a year, maybe 17 quarterly or half a year or something 18 like that. 19 MR. ZACKON: We're going to put 20 out regular minutes after these 21 meetings. Maybe those minutes should be 22 referenced in communication about the 23 site. 24 MS. BURNS: I agree with Pat. It 25 needs to stand alone. 0024 1 MR. ZACKON: It should stand 2 alone. 3 MS. BURNS: Yes. It could be in 4 addition to an A2/M2, but its own form. 5 MR. STERNBERG: Could I make one 6 comment? 7 MR. ZACKON: Please. 8 MR. STERNBERG: I would just 9 think that since the council is supposed 10 to be an independent, quote/unquote, 11 organization, I don't like the fact that 12 Nielsen should be the ones to comment on 13 what the council is doing. I think it 14 should come directly from the council. 15 Nielsen is funding the council. 16 This was set up as an independent. It 17 will look as though Nielsen is in charge 18 of what the council is doing, which 19 isn't really the case. 20 MR. ZACKON: It is a Nielsen 21 client base. We may want to extend 22 communication beyond that. But they're 23 deserving of what is going on. 24 Are we in a position of releasing 25 that to that list? I think we would 0025 1 send it to Nielsen, release of that list 2 and we could have our own list. I would 3 4 and not Nielsen letterhead. That's the 5 dynamics of how you get it out. That's 6 fine. 7 Are we good there? Are you ready 8 to go to the next committee? 9 And Nancy is not here to accept 10 my apologies, so I apologize anyway. 11 Nancy and her committees, RFP, is there, 12 as well. That really sits in the same 13 place. We need to accelerate that. 14 There's no budget against that because 15 we haven't heard from them in terms of 16 what it would cost. 17 I think we're catching up. By 18 the way, we have a backup room across 19 the hall if the pounding from the 20 jackhammers gets too tough in here. So 21 we're hoping not to have to move. 22 Just to quickly go through 23 attendees. Shari, you're here. Tim, 24 Joanne, Michelle, Susan is here. Vicky 25 is on the phone. Colleen is here on the 0026 1 phone. Don is here. Kate is on the 2 phone. Jean is here. George Ivie, Mark 3 Kaline, Rick Keilty, Pat Liquori, 4 Jessica, and Jack are here. Lyle is not 5 here. Ceril and Howard are here. Steve 6 Sternberg and Ira. 7 So that's who is here now. We're 8 ready in that case, unless there's other 9 comments to take on the Ball State and 10 Sequent Partners study, and we have 11 allotted an hour and you have 61 12 minutes. 13 MS. SHAGRIN: I need one of those 14 minutes to ask on the University's 15 estimate since Nancy is not here, it 16 says that the RFP is prepared and ready 17 for release pending final legal review. 18 MR. ZACKON: That's the one that 19 I said this is what a bottleneck looks 20 like. I need to be between the two 21 groups. That's mine. Don't blame 22 Nancy. 23 MS. SHAGRIN: All right. 24 MR. ZACKON: Steve and Shari. 25 MR. STERNBERG: We are actually 0027 1 very excited right now. We are ready to 2 make a recommendation to the full 3 council on whether we should proceed 4 with the full study from Ball State and 5 Sequent the observational study that 6 they recently completed, the pilot 7 study. 8 Shari is going to take us through 9 a very brief intro and then we're going 10 to let Jim Spaeth and Mike Bloxham give 11 an overview of the results of the pilot 12 study and the proposal for the full 13 study which we expect to be followed by 14 a vote. 15 MS. BRILL: Good afternoon, 16 everyone. I am really exited to be here 17 today and because of what I'm going to 18 be talking about and, secondly, from a 19 transportation standpoint, I'm doubly 20 excited to be here. It seems whenever I 21 have to be at a meeting, bad weather 22 follows. It is good to see all of you 23 here and I look forward to sharing this 24 with you. 25 Observations from the pilot study 0028 1 were very compelling. So we unanimously 2 and wholeheartedly recommend to the 3 Council for Research Excellence that it 4 vote to move ahead for the full study. 5 What are we trying to accomplish here? 6 I'll tell you. 7 If you recall when we first 8 formed the council, our committee, 9 rather, the mission of it was to improve 10 above TV and audience measurement 11 through comprehensive and ongoing study 12 of media consumption. I would say about 13 a year and a half ago we circulated an 14 RFP to the industry and specifically our 15 committee sought to one dimension the 16 current conception of media focusing on 17 TV and video, how it is changing and 18 will change over time in order to 19 propose the optimal form of video media 20 measurement. And this was why we had 21 awarded the pilot in the first place. 22 Because we know that asking people to 23 self-report their media behavior use and 24 their usage provides a poor reflection 25 of reality. Increasingly, the amount of 0029 1 actual media exposure is missed by 2 current indicated services. We know 3 that multi-tasking simultaneously usage 4 group versus solitary group occur. 5 There is rampant usage of portable and 6 media use and out-of-home usage of media 7 and that's currently missed or not 8 measured completely. 9 We know that the observational 10 approach will best lead to a blueprint 11 for better measurement, and this is the 12 only approach we have seen that doesn't 13 rely on the past behavior or relies upon 14 electronic measurement of what device is 15 in use. It is going to tell us what 16 people are actually doing at a given 17 moment in time. It captures their 18 behavior as it is happening. 19 So we didn't want to commit 20 $3 million at first without having 21 confidence that the technique was 22 workable. This was what we wanted to 23 do. We wanted to establish a map for a 24 video media usage. What is it now and 25 how it is changing? We wanted to know 0030 1 what it will mean for the future. What 2 are the implications for media 3 consumption and engagement? How will 4 this enhance audience research 5 techniques that are out there? 6 And Nielsen has said that this 7 council's work would help guide their 8 initiatives in A2/M2. Many reasons 9 exist to proceed. None of us can afford 10 to do this individually within our own 11 organizations. This approach will 12 overcome business self-interest because 13 we really put our business interests 14 aside to come together for the common 15 good and advance industry learning. 16 There's no design for anyone to have a 17 competitive edge over anyone else. The 18 media landscape, as we know, is not even 19 just fragmenting, it is segmenting and 20 splintering. Everybody isn't getting 21 every device anymore. So how can this 22 research guide Nielsen to improve its 23 own measurement. 24 Well, first of all, how 25 significant is the media usage that is 0031 1 not currently picked up by Nielsen. 2 This will help prioritize Nielsen's 3 focus. 4 MR. STERNBERG: There are a lot 5 of potential ways that we believe this 6 study can really help improve Nielsen's 7 own measurement. We just came up with 8 three or four off the top of our heads. 9 But once the data comes in, many, many 10 more options, opportunities, I think, 11 will develop. 12 MS. BRILL: One of the key issues 13 is multi-tasking, how much and what type 14 of multi-tasking is actually occurring. 15 Could possibly sole and dual or primary 16 and secondary media usage be separate 17 Nielsen categories for analysis? Should 18 Nielsen instructions on what constitutes 19 viewing change as they instruct their 20 membership panel? How does media 21 ten-second increments different from 22 one-minute increments? Is there an 23 urgency for Nielsen to provide more 24 granule ratings as part of its regular 25 ongoing measurement? 0032 1 So what are we going to be voting 2 on today? We are recommending a 3 two-phase observational study with the 4 potential to review after wave one. The 5 sample will range between some 350 and 6 500 respondents per wave. The cost will 7 not exceed 3.5 million, contingent upon 8 Nielsen funding a third year. 9 MR. STERNBERG: And we thought we 10 might hear as early as today whether, 11 indeed, they were funding a third wave. 12 MS. BRILL: The first wave. 13 MR. DONATO: There is a reason I 14 look like this. 15 MS. BRILL: The second wave would 16 commence October 2008. So to conclude, 17 today is August 8, vote yes, and the 18 statement was provided by the Media 19 Consumption Engagement Committee. 20 And now I would like to turn it 21 over to the representatives from Ball 22 State University and Sequent Partners. 23 MR. BLOXHAM: Thank you very 24 much, Shari. 25 Good afternoon, everybody. 0033 1 Firstly, before I get going, I know I 2 have 20 minutes or we have between us 20 3 minutes to present what we have for you 4 today. And we have, I think, wisely 5 followed the advice of our committee in 6 making best use of this, and we're going 7 to be holding back of going through any 8 detail of the data that was actually 9 captured from the pilot, largely because 10 50 people, clearly, it is not the data 11 that is of value, it is the process of 12 going through this and what we learned 13 from the process. 14 You have already received the 15 report of the pilot which contains the 16 data that we did generate, so that is 17 available to you. Also, before I go any 18 further, it would be remiss not to 19 acknowledge and thank Shari, Steve and 20 the committee for their guidance and 21 help as we run through this. 22 Richard Zackon, from our point, 23 has done a great job as a facilitator 24 and Howard Calvin and the team at 25 Nielsen. Our experience working with 0034 1 them and the guys and girls down in 2 Florida has been exemplary. It would be 3 wrong not to mention that. I'm going to 4 take pot luck which moves forward. 5 Excellent. As you know, it was all the 6 way back when on the 8th of September 7 when the note was finally given to the 8 pilot after the RFP process, and I will 9 not go into detail. 10 The pilot was to be the first 11 step. We were out in the field and 12 applied this method seven times in three 13 years. However, this committee had 14 particular requirements and objectives 15 and they needed to be tested to be sure 16 that this would deliver against the 17 methodology. So we did that test in the 18 single market, Indianapolis. DNA 50 19 person methodologically oriented pilot. 20 The success criteria established by the 21 committee in consultation with ourself, 22 and we set some of our own goals around 23 some of the numbers we wanted to 24 achieve. And those were based in terms 25 of stratifying the sample based on 0035 1 available data about incidence of 2 population and so forth amongst the 3 different groups, and we wanted to try 4 to represent within the sample. 5 Just going through these quickly. 6 What were the overall requirements 7 representing the key target groups. And 8 I won't go into detail. We wanted to 9 maximize, capture their media data 10 through observation member and recall 11 the next date what media they had 12 consumed after the observation. We 13 wanted to deliver the data that was 14 relevant and going to be able to 15 effectively address the issues that the 16 committee is setting out to address. 17 So, fundamentally, this was all about 18 methodological feasibility. Can we do 19 it. The breaking it down, can we 20 recruit the sample that is required and 21 can we capture the data that is deemed 22 of relevance. Does the instrument that 23 we have devised work for this purpose. 24 And then, clearly, any objective, any 25 product, rather, has to carry the 0036 1 objective that we learned from the 2 experience, and we do all that we can to 3 cherry pick whatever learnings there are 4 and to avoid weaknesses that we might 5 identify in any future roll-out or 6 expansion, specifically with regard to 7 the objectives of the study. 8 Well, the good news in the 9 headline, the success criteria met to 10 the satisfaction of the committee. The 11 sample represented the topics of 12 interest. Spanish language dominant. A 13 mix of African-American, inner city 14 subjects. We didn't just want to find 15 that all we could do is recruit more 16 affluent participants, for example, 17 advanced video technology users, and 18 these were defined as having to own at 19 least three of a range of different 20 video enabled devices from DVRs, video 21 capable i-Pods and cell phones. 22 That was one of the criteria. 23 And also individuals from FTO and 24 households. We wanted to have what 25 would be deemed as a robust average 0037 1 observation across all video and media 2 locations. We got an average of 13.5 3 hours of observed time, and then when we 4 took into account the recall data, i.e., 5 the questions that our observers asked 6 about what have you done with regard to 7 media use in the 40 minutes that you 8 tell me you have been awake before I got 9 here. And then the recall data the next 10 day to say after I left, how long were 11 you up for. What media did you use, if 12 any. Where we concluded that the 13 average went up to 16 hours. And within 14 that we got the incidence of use of 15 different media, time spent with those 16 media, where they did that and current 17 media exposure and so forth. 18 Obviously, that wasn't just for 19 video enabled devices or, for that 20 matter, video audio devices. We were 21 capturing non-video devices and 22 functions, as well. 23 Another criteria, the subjects 24 would have to complete the observation 25 in a day. The committee rightly wanted 0038 1 to be sure that we were not rolling out 2 a larger subject where a significant 3 portion of the people were bored halfway 4 through the afternoon. I had enough. 5 Go away after half of the afternoon. We 6 had 50 complete the full observation 7 data. A hundred percent, which is good, 8 and that correlates with our broader 9 experience. 10 And then, obviously, 11 demonstration, collection of useful 12 information on the unobserved latest 13 parts of the day. There was the 14 question, how many people would comply 15 with the requirement to take the call to 16 give us useful information when we 17 called them back the next day. And we 18 achieved all 50 of respondents' 19 participation in that. And the detail 20 there is 48 of them gave us information 21 about their media use, 2 said they 22 didn't use any media after we left the 23 house, which is entirely feasible. It 24 wasn't that they didn't comply. They 25 did. 0039 1 We also found -- I can't remember 2 who came up with the idea. It was good. 3 To say let's try to validate what we get 4 out of this. It stacks up and is 5 logically at least. We did a 6 comparison, a regression analysis 7 against the NSI Indianapolis data 8 looking at Monday through Sunday, 9 average 18-plus half-hour segments, and 10 we got a very good reading. .97 for 11 February and .94 for May, which we were 12 both pleased by and, frankly, slightly 13 surprised by. It is a relatively small 14 sample against a larger sample. 15 Nonetheless, that is the data we worked 16 with and it was encouraging. 17 The other factor was observe a 18 training and quality control. The 19 committee wanted to feel that they were 20 sufficiently confident and had good 21 grounds to be confident in those 22 processes. And both Richard and Shari 23 visited us down in Monsey and saw that. 24 And we took them through the materials 25 and other members of the committee and 0040 1 they were deemed to be thoroughly robust 2 and well managed. 3 The second part was emotional 4 uses and gratifications. Instrument 5 personality profiling needed to be felt 6 to be able to deliver useful relevant 7 information. And the overall conclusion 8 was that obviously, if drawn from a 9 larger sample, it could be valuable to 10 put alongside the observational data. 11 So a key learning out of the pilot to 12 summarize, the data, the process, and so 13 forth, all suggest, really, if similar 14 data from a larger sample, a projectable 15 national sample, this process when 16 scaled up would squarely address the 17 objectives of the committee. 18 The real additional learning and, 19 for my money, the greatest learning, I 20 guess, because we have gone through this 21 process a lot of times and we felt that 22 was the top line of the first paragraph, 23 we weren't surprised by that. But this 24 was the stuff that was really valuable 25 to us. This came out of the 0041 1 collaboration with Nielsen. We got a 2 lot of learnings about the recruitment 3 process and we were aware and we will 4 talk about it at the last meeting, there 5 was a lot of discussion about response 6 rates before the information was in. We 7 have the information and Jim will take 8 you through that. We took learning out 9 of that process through collaboration 10 with Nielsen and also consultation with 11 Professor Grove's groups and the 12 University of Michigan. Took their 13 input to the whole process to define 14 ways in which not only we might want to 15 view the whole issue of response rates, 16 but also we can take steps to improve 17 response rates and get a level of 18 confidence about what we could achieve 19 going forward. 20 With that in mind, I'm going to 21 hand over the speaking to the taller, 22 better looking, Mr. Spaeth. 23 MR. SPAETH: In this way, as Mike 24 just said, this is the new learning for 25 us. This technique was never done for 0042 1 us. The seven times previously it was 2 done, all done on a purpose of sample 3 recruiting specific groups for specific 4 purposes. And, in the best of our 5 knowledge, searching the literature, no 6 one has reported on doing an observation 7 study on a random basis. All the 8 methodological learning that we built 9 over decades or telephone research 10 doesn't apply here. 11 We do know as the task becomes 12 more challenging, longer interviews, 13 harder to recruit segment, consumer 14 segments, that response rates tend to 15 decline. This is a pretty amazing 16 request of the respondent, but, 17 nevertheless, we have people do it. 18 Here is the way the process 19 works. There is a pre recruit mailer 20 sent by Nielson which explains the 21 study. Telephone recruit for the 22 initial agreement, the calls made by the 23 Nielsen RIs. Then the agrees are passed 24 on to Ball State for scheduling. 25 Three-step process. 0043 1 Top line, what did we find. We 2 found that 47 percent, just short of 3 half of the predesignated sample, were 4 not contacted. One of the reasons, it 5 was quite a while since they were in the 6 Nielsen sample. The telephone numbers 7 were not probably still accurate and so 8 forth. And a number of them not 9 contacted because at a certain point the 10 sample size was filled and we didn't 11 continue to make phone calls. Take half 12 off the table to begin with. 13 Of the hundred percent, 43.8 14 percent did not agree to be contacted by 15 Ball State. One of the reasons for that 16 and possibly the key reason for that was 17 Nielsen's own best practices and 18 recruiting were not applied. There was 19 a legal concern about liability, and 20 Paul can speak about it at length, I'm 21 sure, or not. I'm just kidding. There 22 was some real concerns and there was 23 really no persuasion involved in the 24 recruitment process. With a very plain 25 vanilla, did you get the letter, would 0044 1 you like to participate, we'll give your 2 name to Ball State. 3 MR. DONATO: When we recruit for 4 the very first time, there is an element 5 in the recruiting script, we will not 6 give your name to anybody and you will 7 not be recruited for other purposes. So 8 legal counsel said we have an obligation 9 to contact the people first to ask them 10 if they wanted to be contacted. 11 MR. SHIMMEL: Other issue is that 12 we're concerned about playing a harder 13 role than selling cooperation. It is 14 not a Nielsen employee who is doing the 15 observation. So we're concerned about 16 the liability if something were to 17 happen where somebody's DVR got broken, 18 who would bear that responsibility. 19 MR. SPAETH: Right. I did not 20 say a non-legitimate concern. There was 21 a concern. There was little persuasion 22 applied here. 47 percent we didn't 23 contact. 48 refused. The good news of 24 those names that were passed to Ball 25 State, 77.8 percent agreed to be 0045 1 scheduled and were observed. 2 Here is the way the numbers break 3 down. If you want to take the harshest 4 possible look in the interest of 5 transparency, the harshest possible look 6 would be here is a total predesignated 7 sample. Here is the 35 people 8 interviewed. 9 There were 15 hours interviewed 10 by Ball State using different 11 techniques, ten high-tech users and five 12 Spanish language dominance. Of the 35 13 recruited, you calculate a frightening 14 4.5 response rate. It is reasonable to 15 say those with bad phone numbers 16 wouldn't be included, and I think 17 because the recruitment was cut off 18 before completed, I would say take the 19 non-contacts off the table. 20 Working with the people, Nielsen 21 was able to contact -- we had quite a 22 few who refused for the reasons we 23 talked about, which gives you 7.8 24 percent response rate. Still not 25 fabulous. And, again, to look of those 0046 1 that Ball State was able to connect 2 with, 77.8 percent agreed to be 3 scheduled and, for the most part, well, 4 half roughly, more than half of the 5 people who refused Ball State, I was 6 busy at the time, there was an 7 indication with more flexibility we 8 could schedule them more successfully. 9 That gives you cause for pause. 10 We were happy when Nielsen did a 11 nonresponse study. These are former 12 Nielsen cooperators. We have the 13 historical data. You can ask yourself 14 was there any difference in the 15 historical viewing data. According to 16 the test Nielsen applied, there was no 17 bias found. There was no clear bias 18 between cooperators and non-cooperators. 19 That said, I have to point out it is a 20 small sample. It would have to be a 21 strong bias to observe one. 22 The check that was able that 23 could be applied at this point showed no 24 reason for concern in terms of bias, but 25 that's not really enough. Again, this 0047 1 was the real learning here about first 2 time having done this on a random basis. 3 So now what do you do. There was a lot 4 of consultation with Nielsen. We went 5 to the University of Michigan, spent a 6 day with Bob Groves and his Ph.D. 7 students and came up with a number of 8 ideas. Some obvious, but some not so 9 obvious. 10 So recruit the former Nielsen 11 cooperators closer to their date of 12 turnover so we have more good numbers 13 and the Nielsen experience is fresher in 14 their minds. Increase incentives and 15 you'll see in a moment the significant 16 of increasing in incentive. Use Nielsen 17 equity with respondents. There is more 18 clearance to be more persuasive and 19 refer to the Nielsen experience in the 20 process of recruitment. Permit the RIs 21 to have written persuaders. Here is 22 what they have if they have concerns or 23 questions, to try to be more successful 24 in the recruitment. And Ball State to 25 be more flexible in scheduling so people 0048 1 who have a busy week, you don't say, no, 2 I'm sorry, but you say, how about next 3 week. 4 The thing not obvious, and this 5 one takes a minute to get your head 6 around, was Bob Groves' biggest 7 contradiction was, in my opinion, to use 8 the Nielsen sample for its strength and 9 to use the Ball State data for its 10 strength. We have a high quality sample 11 to start with, and so, we have changed 12 the sample design now to stratify the 13 sample by known drivers of viewing, by 14 gender, by age, probably presence of 15 children. We're still thinking about 16 those. Household size, one of those 17 factors that we know drives viewing 18 data. So when you have the sample 19 strata, you're picking from the fully 20 enumerated sample. You can pick. You 21 minimize your concerns about bias and 22 drawing from a high quality sample to 23 begin. Then bring in the Ball State 24 data within the context of that sample, 25 the richness of the data collected. You 0049 1 have measured occasions which are 2 observed. So occasions, viewing 3 occasions side by side with the Nielsen 4 metered viewing, so you do a number of 5 the analysis that Steve and Shari talked 6 about. You can understand unmeasured 7 occasions, what are the occasions when 8 people are using video and whatever 9 platform, an iPod or cell phone or 10 television somewhere else. That is 11 simply not being measured. And you can 12 understand not just the viewing 13 occasions current measured or unmeasured 14 but the value of those occasions. 15 I will mention, not going to take 16 the time to explain and it is awkward, 17 it is in the handouts that were sent out 18 in advance, and that is a number of ways 19 to understand the value of the exposure. 20 But one, I am particularly fond of, the 21 six degrees of behavioral engagement. 22 It is looking at whether the moment of 23 exposure was only watching video, 24 watching video while were engaged in 25 some life activity, watching the video 0050 1 while you're involved with some other 2 media. Whether you're giving video your 3 primary or secondary attention and also 4 whether there is life activity going on 5 at the same time. You can partition all 6 these viewing experiences into six 7 different types of classic situations. 8 One, television or video is the 9 only thing you're doing. You would 10 expect, and it is just an expectation, 11 you would expect high attention. The 12 other extreme, you're washing the 13 floors, the television is -- you're 14 reading a magazine, the television is 15 fully in the background. It is getting 16 very little attention. You can start to 17 understand not just the number, the 18 quantity of occasions measured and 19 unmeasured or the nature of the 20 relationship between the two measured 21 sets of data, but also the value of the 22 exposure, we thought that was very 23 important. And when you think of this 24 being the media consumption, that's the 25 occasions and engagement, that's the 0051 1 potential step. 2 Anyway, okay. So what do we 3 propose going forward is a prerecruit 4 mailer, a letter from Nielsen that 5 leveraged the Nielsen experience. 6 Provide more info about Ball State, but 7 with an insert so more information or, 8 frankly, more information pictures of 9 the observers in action. Mentioning 10 those who participated in the past 11 enjoyed it. Quotes from the pilot 12 participants. And these are based on 13 Nielsen recommendations which we think 14 are great and will really tap expertise. 15 A $30 incentive would go in the 16 prerecruit mailer. That is an 17 additional expense, but will get some 18 attention, to some degree cooperation at 19 that step. 20 The telephone recruitment. The 21 Nielsen RIs will leverage the Nielsen 22 relationship. Prepare persuaders for 23 common questions. Cite satisfaction for 24 previous participants. The incentive 25 went from $200 to $300 total. So it is 0052 1 a 50 percent increase. That should 2 help. $500 for initial refusers, and we 3 budgeted for that for about 20 percent. 4 I think between the prerecruit 5 incentive, the 50 percent increase in 6 incentive, the 150 increase for initial 7 refusers, plus the ability to leverage 8 some of the Nielsen relationship. I 9 think it is fair to say an improvement 10 in response rates. 11 We don't know what it will be, 12 but we'll learn a lot as we go along. 13 At the Ball State step we can schedule 14 observations with greater flexibility 15 and check to ensure that the data that 16 they have selected is expected to be a 17 typical date. It is not Super Bowl 18 Sunday or anybody's wedding, but a 19 typical day. The rest of the procedure 20 as we have done in the past, reminder 21 calls and so on. This is an experiment, 22 a mind experiment, a thought experiment. 23 This is not to be a suggestion of what 24 would happen, but if you said for a 25 moment what might happen. If there was 0053 1 a 10 percent reduction in refusals to 2 Nielsen, and maybe that is ambitious, I 3 don't know, here is the response rate we 4 started at 7.7. 10 percent improvement. 5 At the scheduling step it could take you 6 to a response rate just north of 7 20 percent. 8 I don't know if that's going to 9 happen and I'm not going to say that it 10 will not happen. It is a first step of 11 a number of experiments as we really 12 learn how to do this in a random 13 process. There is a lot of learning 14 here, but I think we have good 15 ammunition to include the situation. 16 I want to sum up the pilot 17 experience. We met the predefined 18 success criteria, the observation and 19 survey data. We didn't talk about the 20 survey data, but there is a uses and 21 gratifications and personality profiles 22 and so on. We have demonstrated in 23 pilot data which you had that they 24 offered the potential to squarely 25 address the media consumption and 0054 1 engagement commitments objectives when 2 taken from a larger sample. 3 We have clear steps that we agree 4 that will lead to some degree to higher 5 response rates, can't say what. We have 6 input from the survey research center, 7 led us to revise the sample logic and 8 use it for what it is best for. And the 9 Ball State data, what it is best for. 10 We have to say, as Mike said, we had 11 extraordinary cooperation from Nielsen 12 and that abodes well for success. 13 So what do we propose to do? 14 Here is the objective, if you remember 15 that. We propose a number of 16 deliverables. These are taken off the 17 RFP and we can do that and do that and 18 that. The clear and important mapping 19 consumers' use of time and, again, it is 20 in the pilot report. A map of the day 21 media used, reach of each media and time 22 spent with each media, and broken out in 23 any number of ways. Simultaneous media 24 used. Observed not remembered. 25 Multi-task with non-media activities and 0055 1 the cross between those two, broken out 2 by as many ways as the sample size will 3 permit, which is an important question. 4 Broadband access and other technology 5 dimensions, ownership or access of newer 6 technologies, as well. We want to 7 understand the way this is influencing 8 behavior. Do we not break it down as to 9 people with DVR's for two years, people 10 one to two years, people that don't have 11 one. How does the behavior, sample size 12 permitting, we can do all of those 13 things. 14 Another thought there is to 15 measure the same people over time. This 16 proposal has the same respondents 17 measured twice in different points in 18 time. We're basically beginning to 19 track the same people over time. If we 20 keep doing that, we would have clear 21 within subjects comparison of how 22 individual's media usage changes over 23 time. We're looking through the survey 24 instruments, media satisfaction and 25 engagement. I mentioned the 6 degrees 0056 1 of behavioral engagement, solitary 2 versus social viewing, consumers' 3 motivations for viewing. Pretty much 4 everything you asked for. The shift, 5 time shifting, using of technologies, 6 advertising related behavior. Because 7 we're coding in increments whether the 8 consent is ad or promotional content, 9 promotional content and on the issue of 10 how each component might be measured. 11 The survey is going to really map out 12 all this happens, how much of it happens 13 and how important it is and a lot of 14 detail how consumers are using it. It 15 is not going to tell us what the 16 technology should be exactly for 17 measuring it. I think it will point to 18 what is necessary and the right question 19 is always more than half of the -- 20 MR. BLOXHAM: That's what we know 21 about once we start digging into the 22 data, we know there will be other stuff, 23 as well. 24 MR. SPAETH: We will be looking 25 at personality types, uses of 0057 1 gratifications, and how that impacts 2 media choice. Tim's idea, looking for 3 projections, looking at cohort analysis. 4 And I said if we were to measure the 5 same people over time, we will do it 6 twice, if we continue that, we will have 7 a very clear trajectory of adoption and 8 behavior issues. 9 To get very specific and, again, 10 it is in the proposal document that you 11 have has these details. There has been 12 some work with the committee and I 13 wouldn't pay attention to the costs in 14 the back of the book. We're working 15 through those details and Shari showed 16 you what the proposal is for today, so 17 don't be too concerned with that. We're 18 looking at 17 different life activities. 19 These are meant to fit into the American 20 time use study, and that provides some 21 interesting opportunities. 22 Measuring use in all locations 23 coded into seven categories; home, car, 24 public transit, work, school, home, 25 other home. Guest viewing as a 0058 1 measurement of viewing in others' homes 2 and other locations for whatever is left 3 over. Seventeen media, we're capturing 4 all video, DVR, VCR, web, which would 5 include video, streaming video on the 6 web, computer media, which would include 7 video, mobile phone, portable video like 8 an iPod, digital transfer uploading, 9 downloading cinema. I am not sure 10 whether you want to include that. We 11 want to capture the full picture in 12 which these media fit. 13 You might be interested in where 14 is it coming from or where is it going. 15 That would be helpful, as well. These 16 are just -- the top lines there are up 17 to seven subcategories within. 18 Self-report, that's the observation 19 data. The response fill-in 20 self-administered questionnaire and then 21 send back. Including permits, uses and 22 gratifications and emotional uses, as 23 well as traditional uses and 24 gratifications. 25 Just to wrap up in the seconds 0059 1 remaining, if you stop and think about 2 this, this is really an interesting 3 first. We'll be working with the 4 committee on the exact scope and shape 5 and size, but I think even the smallest 6 proposed sample that we might conduct 7 would deliver a truly unsurpassed -- and 8 we threw that word around a lot -- 9 consumer concentric, that's thrown 10 around a lot. I don't know anything 11 more than that data based cross platform 12 video audience measurement. And we 13 thank you for the opportunity. 14 MR. ZACKON: On the phone, we're 15 communicating on the phone with Vicky 16 and Kate. 17 MS. RUSH: Yes. 18 MR. ZACKON: Very good. Thank 19 you both. 20 Thank you, Jim and Mike. And 21 there's time for questions. 22 MS. SHAGRIN: Would the cost be 23 lower, by lower meaning not just a 24 couple hundred dollars, would the cost 25 be lower if instead of trying to do this 0060 1 nationally, you use the local people 2 meter samples which are part of the NTI 3 sample, and you did this in New York, LA 4 and Chicago? That gives you east coast, 5 west coast and central, it gives you 6 America with a lot of public 7 transportation. One with long driving 8 communities and one with a sort of 9 midwestern middle of the road. Thinking 10 about having enough sample, we would 11 have it in those markets. And I thought 12 it might be a way to do all this. Still 13 have the same sample size, but perhaps 14 reduce costs because the travel is so 15 much a part of this. 16 MR. SPAETH: You are either a 17 mind reader or a step ahead. That is 18 exactly the discussions this past week. 19 The sample has to be clustered. You 20 can't manage an observation study that's 21 dispersed in that kind of sense. You 22 need a small number of sampling 23 clusters. And we are in the midst of 24 thinking about the markets because we 25 want people meter turnover folks, as 0061 1 well. 2 MS. SHAGRIN: Those three markets 3 give you three very different types of 4 capability. It gives you the urban, it 5 gives you apartment lovers, it really 6 gives you Blacks, Hispanics. Just gives 7 you a very broad range, even income 8 wise, which is why I'm suggesting -- 9 MR. SPAETH: We will work that 10 out with the committee. The idea is so 11 good, it has aroused the poltergeist 12 above us. 13 MR. ZACKON: With a jackhammer. 14 MR. IVIE: Firstly, I want to 15 thank you guys for your straightforward 16 presentation of the components of 17 response rate to your study, to your 18 pilot study. I appreciate that. The 19 question I have is not related to 20 response rate, so you will be happy to 21 know that. 22 In using the Nielsen sample as 23 the frame essentially for your 24 selection, the national sample you get 25 kind of saddled with Nielsen's success 0062 1 in installing councils that have certain 2 technologies, and I noticed one of the 3 components of your analysis, I wrote it 4 down, is ownership or access to new 5 technology. And so since you're looking 6 at turnover in Nielsen households, 7 Nielsen's ability in the past maybe a 8 year ago or more to meter those 9 technologies plays into whether you will 10 observe them in your sample. I am a 11 little concerned about that. I'm 12 talking about the details of that. I 13 think that's something you need to talk 14 to Nielsen about. 15 MR. DONATO: It is easier to 16 number, what the number of DVR samples 17 in the home. 18 MR. IVIE: How many DVR homes did 19 you have in your 50 sample because 20 those were prior Nielsen homes. 21 MR. SPAETH: Right. We also 22 indicate, we didn't go into the details 23 because of the time, but we recruited 24 ten, the committee specifically wanted 25 us to recruit ten high-tech individuals. 0063 1 We had a list of devices from DVRs and 2 iPods and video enabled cell phones and 3 so on. There had to be at least three 4 of that list to qualify as high tech, 5 and they were recruited on a purpose 6 basis just to make sure we had those 7 people to measure their activities. 8 It is a good point that we need 9 to think about what we might be missing 10 and do we need to supplement the 11 benefit. The great benefit of drawing 12 the Nielsen sample, generally what we 13 consider very high quality, is that we 14 have the Nielsen measurement to utilize 15 in the analysis, as well. 16 MR. BLOXHAM: We have the data of 17 the Nielsen households that had DVRs and 18 other devices. 19 MR. IVIE: In the 30, 35, 35. 20 MR. BLOXHAM: Yes. 21 MR. IVIE: Are you proposing to 22 incorporate the side sample of the homes 23 or does that go away? 24 MR. SPAETH: We were thinking it 25 would go away. That is a topic the 0064 1 committee could consider. 2 MR. DONATO: It is lot less 3 necessary, giving the timing that we 4 went into field with A P started 5 measuring your DVRs. I expect by that 6 time you would get into field, it might 7 not be an issue anymore. It is easy 8 enough to document. 9 MR. SHIMMEL: George, your point 10 is a valid point. I think part of the 11 trade-off we're going to have to deal 12 with is, you know, how tightly 13 clustered, how many markets, is it 14 something like Ceril recommended, or how 15 fresh is that sample. And, you know, it 16 is balancing and I think one thing we'll 17 have to know is that if we need to go 18 back in time, and we still may need to 19 go back 12 months, 18 months, two years, 20 just to meet the sample sizes we need. 21 If we get a 15 percent response rate and 22 we're looking for 500 completes, we need 23 to start with about 3,000 households. 24 But I think that becomes a 25 checkpoint that if we have an FTO home 0065 1 from two years ago and it is the dollars 2 because of DVRs, that should not 3 qualify. So I think your point is very 4 valid. 5 MR. IVIE: I think you can get 6 around it, but I want to make sure 7 you're thinking about it. 8 MR. POLTRACK: When you do 9 research of this scope, one of the tests 10 you spent a lot of money on research is 11 whether the results are accurate. I 12 guess it is a question more for the 13 Nielsen representatives. You have seen 14 the design of the study and you have 15 seen the results of the pilot. Do you 16 feel that there's something there that 17 is actionable or if it holds up in the 18 full study. And looking at it from your 19 perspective, looking at the study, do 20 you see specific objectives or goals of 21 Nielsen that might be served by the 22 study. 23 I just don't want this to be a 24 study that makes a great ARF 25 presentation and gets New York Times 0066 1 advertising column and then goes away, 2 and that's really going to be Nielsen's 3 initiative that's going to determine 4 that. I want to know what is the 5 Nielsen's perspective. 6 MR. DONATO: Everyone is familiar 7 with A2/M2, and I think that detailed 8 this as sort of a quasi quantitative 9 study with very close observations, very 10 useful information. I think that the 11 $64,000 question is, you know, would we 12 look at other methodologies to kind of 13 round out the picture. So the answer is 14 yes and no. 15 We have looked at results of the 16 study. I think the understanding, the 17 duplication and behavior and 18 cross-platform is very important. We 19 would probably look at other 20 methodologies, to be honest, to 21 understand from a more quantitative 22 perspective what was going on out there. 23 But, yes, I can see how we could react 24 to this. 25 MR. WAKSHLAG: One of the 0067 1 thoughts that the committee had was if 2 this is funded by us and we're in four 3 markets, there's nothing that precludes 4 Nielsen from feeling about a study of an 5 alternative possible methodology. You 6 know, I'll make it up, a personal diary, 7 that's not what I mean. 8 MR. ZACKON: Good choice. 9 MR. WAKSHLAG: They could pick 10 other alternative methodologies. Phones 11 to lay a parallel to this to see what 12 happens. This gives an interesting base 13 at other methodologies, you can look at 14 various response rates, you can look 15 patterns to see if there is a 16 consistency or inconsistency at the 17 data. If there is consistence and it is 18 much cheaper, that's great news. 19 MR. DONATO: We're going to have 20 to take care of that noise. 21 I don't think that we would use 22 personal diaries. Electronic diaries 23 more likely. We have done a lot of 24 other investigation in the other 25 industries. Pharmaceutical industry has 0068 1 made progress in terms of use of time 2 kind of data in switching over from 3 paper diaries to electronic diaries. 4 PDAs with prompts go a long way to 5 solving the glitches that are inherent 6 in current paper diaries. You read my 7 mind kind of, Jack, without getting into 8 too much idea, we have ideas of how we 9 might do something, but from a slightly 10 more quantitative point of view, I would 11 describe this as a kind of half 12 qualitative and half quantitative. And 13 there are other more quantitative ways 14 of doing it. 15 You look at it like you look at 16 focus groups and survey research. Focus 17 groups, you're there, you're asking 18 detailed questions, you know how to put 19 the results into perspective. And 20 survey research has its limitations, but 21 it comes about the same questions from 22 another angle. 23 When I said before that we would 24 like to look at multiple methodologies 25 if you were to go ahead with this, we 0069 1 would look at this as that sort of 2 detailed information, highly granular. 3 I don't want to call it a focus group. 4 It is more quantitative than a focus 5 group. And we would look at that for 6 that perspective, and then we very well 7 might do something like Jack suggested, 8 and that is having the more quantitative 9 thing. 10 MR. ZACKON: We allocated an hour 11 for this piece and 15 minutes to 12 deliberate it if we were to begin now. 13 So I want to make sure, having been 14 informed there's other questions of Ball 15 State or Sequent, now is the time to do 16 it. 17 MR. POLTRACK: One more question. 18 The Ball State people, full 19 participation, and I guess the one 20 reservation I had about the technique 21 which evidently did not show up in the 22 pilot is the cooperation in the 23 workplace with the employer of the 24 people, and you did not run into any 25 situation where your access to a 0070 1 workplace was blocked or partially 2 blocked. 3 MR. BLOXHAM: I believe that's 4 correct, Michael. Do you want to -- 5 MR. HOLMES: We had good 6 cooperation from employers' workplaces 7 in our previous studies. We recognize 8 that this may differ slightly in parts 9 of the country. There are certain kinds 10 of workplaces like surgical theaters, 11 that sort of thing. However, one way 12 that's dealt with, a person says I work 13 in a foundry, insurance reasons, you're 14 never going to get in the door. We say 15 can we interview on a weekend day. We 16 balance that against our observation 17 across the day. There is certain 18 places, workplaces where we are unlikely 19 to get cooperation. I will have to say 20 I have been remarkably pleased in the 21 various studies that we have run with 22 the cooperation we have received so far. 23 There was an airline pilot within the 24 sample that we observed this time 25 around, obviously not in the cockpit and 0071 1 not on a day of work. We got that level 2 of affluence and that profile. 3 MR. BROOKS: It doesn't sound 4 like instances there is media use. 5 MR. BLOXHAM: I would hope not. 6 MR. POLTRACK: They're always 7 listening to music. 8 MR. BROOKS: Including from a 9 place where media is, have you run into 10 anybody? 11 MR. HOLMES: We had so few cases 12 where we have not worked out either 13 access to the workplace or the person 14 volunteers. We don't know why they said 15 we would like to be observed on a 16 weekend, but it might be because of the 17 nature of their workplace. 18 So, we really don't have a pool 19 of workplace refusers to draw upon to 20 say here is the kind of workplaces we're 21 not going to get in to. 22 MR. BLOXHAM: Which is good news. 23 MS. SHAGRIN: It might be in the 24 data that we have. Of the 50 people 25 that you had, were they equally 0072 1 distributed across the seven days of the 2 week? 3 MR. SPAETH: Yes. Within five to 4 nine. Between five to nine each day of 5 the week. Seven is the average. We 6 gave leeway, but they were distributed 7 in that proportion. 8 MR. HOLMES: We found one of the 9 harder days to recruit from and one of 10 the days that had nine is Sundays. 11 MR. SPAETH: Five. 12 MR. HOLMES: Yes. Five for that 13 day. The issue there is, is that one 14 place where you exercise your greater 15 incentive to make sure that you have 16 that. It is a very important day for a 17 number of media; so, we want to make 18 sure it is adequately represented. 19 MR. IVIE: I have a question but, 20 first, I would just say I think the 21 observational issues you have at the 22 workplace are probably coming out in the 23 4 or 7 percent response rate, you just 24 don't agree to participate in this if 25 they work in a place that is difficult, 0073 1 I would think. I'm not sure, but that 2 would be my suspicion. 3 My question, I want to go back. 4 I thought David's first question was 5 particularly important in terms of how 6 Nielsen might learn from this data. 7 The way I'm taking away, Paul, that was 8 kind of a luke warm answer. You didn't 9 seem very excited. I was expecting you 10 to say we're studying meters and this 11 might give us some information about how 12 people use small electronic devices 13 during the day that we could learn from. 14 I didn't hear that necessarily. 15 MR. ZACKON: Before you answer, 16 do we want to have this conversation 17 with the vendors present or can we 18 continue this conversation within the 19 council? 20 MR. IVIE: It doesn't matter to 21 me. 22 MR. ZACKON: My sense was that we 23 deliberate and vote amongst ourselves 24 recognizing that we're being recorded 25 and there will be a transcript. 0074 1 MR. DONATO: I have a proposal in 2 the interest of elevating research 3 principles, not vendors. 4 MR. ZACKON: Research partners. 5 So are there other specific 6 questions of Jim or Mike or Bill or 7 Michael? 8 MR. POLTRACK: One other thing. 9 The original plan that there was going 10 to be an attempt to recruit outside of 11 the Nielsen. 12 MR. SPAETH: That was just the 13 pilot. 14 MR. POLTRACK: In the pilot. 15 And it was aborted, I guess. Was that 16 because it was found to be not workable? 17 MR. SPAETH: No. We recruited 35 18 participants through Nielsen using a 19 random process. We recruited -- Ball 20 State recruited 15 directly using the 21 techniques they have been using to get 22 the ten high tech and the five Spanish 23 language dominant. You answer in a 24 vague way, can they be observed 25 successfully with the language issues or 0075 1 all the technology. 2 MR. POLTRACK: I guess I'm going 3 to -- obviously, is the issue we're 4 making a big concession by using the 5 Nielsen, all these cooperation rates are 6 taken on the fact that the initial list 7 is cooperators to begin with. So by not 8 going outside the Nielsen cooperator 9 base where that is a major compromise of 10 the study, was there any experience 11 trying to do that, to go to incorporate 12 some sample outside of the Nielsen base? 13 MR. MOULT: Can we get a better 14 sample with Nielsen and non-Nielsen 15 individuals for theory? 16 MR. POLTRACK: One is basically 17 yes, that is as the first part. The 18 second part is even if we recognize that 19 the Nielsen cooperators solves problems 20 and makes the study more efficient, 21 should we do a separate mini sample or 22 something outside of just not, you know, 23 if we're going to spend this much money, 24 why not get a sub-sample of people 25 outside the base just to see if there's 0076 1 any significant differential. 2 MR. MOULT: We could do that in 3 design. Our opinion was the best sample 4 is with Nielsen households or 5 individuals. And in some ways this 6 dovetails with the other project that 7 you're doing on nonresponse which 8 addresses, separately addresses that. 9 It is conceivable to include people but 10 we don't think it will improve the 11 strength of the sample. 12 MR. DONATO: I would agree. This 13 is a cold call. 14 MR. WAKSHLAG: They did seven 15 studies based on cold calls. 16 MR. HOLMES: We recruited 412 17 successfully, 394 successfully, but we 18 only report data on the longest 350 19 days. Because at that point we had not 20 refined our measurements for the other 21 parts of the day. 22 We have experience doing that. 23 It is a different beast from what we 24 proportioned here. 25 MS. BRILL: I would like to add 0077 1 something here. One of the advantages 2 of recruiting from the retired Nielsen 3 sample is that we can also benchmark the 4 observations against their historical TV 5 and video usage. 6 MR. ZACKON: Do you have a 7 concern that we have sufficient time 8 amongst ourselves to discuss and decide 9 the issue? 10 MS. BURNS: I want the room to be 11 comfortable in getting any questions 12 asked. 13 MR. ZACKON: Questions we have of 14 our four research partners we can 15 research. The first one is coming back 16 is Paul. 17 Are there other questions? 18 MR. SHIMMEL: I have one 19 question. This sort of gets to the 20 point of if this study were to be 21 approved, do you want this to be random 22 projectable to the average adult, 18 23 plus, or something different? 24 Do you have a sense from the 25 data? I know it is only 50 people. 0078 1 There are certain people by the nature 2 of consumed are very TV centric. There 3 are other people very out-of-home device 4 centric, and it seems like the more -- 5 if there were a way to screen for this 6 end of the pendulum, that would yield 7 much more insight than paying money to 8 interview people who 95 percent of the 9 time they watch TV and in home and 10 they're captured in technology. 11 Can you address that through this 12 study or maybe through some of the other 13 observational research you have done? 14 MR. HOLMES: Our segmentation, 15 you do see that that segment of the 16 person who is heavily at home watching 17 TV and you already know about that. 18 I'll leave it to the rest of the team to 19 comment to make sure that we don't 20 over-balance. 21 MR. SPAETH: One of the things 22 that we saw in the sample, we recruited 23 ten high-tech folks but total 50 sample. 24 We saw a significant amount of 25 cross-media use, cross-video usage, 0079 1 significant amount of out-of-home 2 viewing and usage. You know, it is hard 3 to say what the benchmark should be. We 4 showed usage on video iPods. No usage 5 on video cell phones we have to say, but 6 I'm not surprised at that. We did see 7 portable DVD players. We saw a wide 8 array of different kinds of media usage 9 in that sample. 10 MR. HOLMES: It is in the report 11 that you received, the breakout. 12 MR. STERNBERG: Can we add five 13 or ten minutes to the deliberation 14 process? 15 MR. ZACKON: We already have. 16 What I'm trying to do it to make best 17 use of these four experts. 18 MR. KALINE: This is the time to 19 do it. 20 MS. SHAGRIN: I have one question 21 that sort of gives some added value if 22 you can do this. 23 You were monitoring or observing 24 and recording on a second-by-second 25 basis. 0080 1 MR. SPAETH: Ten seconds. 2 MS. SHAGRIN: Will you be able to 3 take your data for these observations 4 and match that actual ten-second data -- 5 do you know what they are watching? 6 MR. SPAETH: We reported it as 7 program or watchable. 8 MS. SHAGRIN: You are reporting 9 it as program or commercial? 10 MR. SPAETH: Right. Not the 11 exact. 12 MS. SHAGRIN: If we have an exact 13 time where we could match it, it would 14 give us some significant insight as to 15 how people actually watch commercials or 16 change channels or don't watch 17 commercials on a far more granular basis 18 from anything we looked at before. 19 And when I think about how we 20 would use this data and how Neilsen 21 would use this data, that's been my 22 concern from day one as to how valuable 23 is it. If we could do that, that adds 24 some new value to this data. 25 MS. BURNS: Jim, if I recall 0081 1 correctly, you were going to record 2 program or commercial not specific 3 because we were afraid of altering 4 habits because we were letting, I'm not 5 recording the actual show, just that 6 you're watching a program. 7 So we can get your answer and 8 that was the intent, just not on a 9 specific program basis because we want 10 them to watch naturally. 11 MR. BLOXHAM: One specific 12 breakout is putting that information 13 alongside the information relating to 14 distribution of attention, so when 15 somebody is visually fixated on the 16 screen, we know they're watching. On 17 the other hand, if they're looking at a 18 newspaper and alter attention we can 19 record that. 20 So you would have a breakout for 21 program versus advertising time against 22 that kind of measurement, that's one 23 thing that comes to mind as an output. 24 MS. SHAGRIN: But you wouldn't 25 necessarily know that they saw ten 0082 1 seconds of this commercial and then 2 changed to a different channel? 3 MR. BLOXHAM: If basically it 4 came out of a programming into 5 advertising, that would be captured. At 6 some point they switched channels and 7 went into programming, it would be 8 recorded as programming. 9 MS. BURNS: It is generic 10 program, generic commercial, as opposed 11 to specific. 12 MR. BLOXHAM: If after 20 seconds 13 they flipped. 14 MS. GALLAGHER: Would we know it 15 was a flip or 20 seconds worth of 16 commercial? 17 MR. BLOXHAM: We wouldn't know 18 for sure. 19 MS. SHAGRIN: You're not going to 20 get what you want. 21 MR. BLOXHAM: We have conducted a 22 separate study which got into that level 23 of detail to ask that of one, any one 24 media in a study. Looking at so many 25 different media is overly burdensome on 0083 1 the observer and, frankly, compromises 2 our ability to capture the data. 3 MR. SPAETH: It is another study. 4 MS. SHAGRIN: Fine. 5 MR. HOLMES: Behavioral levels of 6 engagement and how those are associated 7 with program content versus ad content, 8 that is possible. 9 MR. ZACKON: Other questions for 10 our research partners? 11 MR. BLOXHAM: Propeller heads, 12 please. 13 MR. ZACKON: Anyone on the phone? 14 MS. BURNS: Are you still on the 15 phone? I'm wondering if they're on the 16 phone. 17 MR. ZACKON: I personally want to 18 thank our four research partners. We 19 have grown close working on this 20 project. Thank you for excellent work 21 on this, on the preparation. 22 MR. BLOXHAM: Thank you. 23 MR. SPAETH: Thank you. 24 MR. MOULT: Thank you. 25 MR. HOLMES: Thank you. 0084 1 MR. ZACKON: I assume that 2 presentation we can e-mail out after 3 today's meeting. 4 MR. BLOXHAM: On the back there 5 are appendices of that data that you 6 have in the white data which is attached 7 at the end, you have access to that. 8 MR. ZACKON: We have our copy of 9 the current proposal. So it is time to 10 discuss the formal motion. If you want 11 to repeat the formal motion, Shari or 12 Steve. 13 George, you had a question of 14 Paul. You're not off the hook. 15 MR. DONATO: I thought we 16 restored order. 17 MR. IVIE: It is not a hook. 18 MR. DONATO: In the last minute I 19 kind of jotted down six dimensions that 20 I think is probably worth considering. 21 One is a sample representation -- it is 22 what it is. I would assume that we may 23 come in with about a 10 percent response 24 rate, maybe, and, you know, whether it 25 is 5 or 10 percent, you have to sort of 0085 1 take this kind of information for what 2 it is. Just like you take focus group 3 information. It is one view of the 4 data. 5 There's some diagnostics that we 6 can do that will help us understand 7 whether or not the sample is leading us 8 in the wrong direction. But, as I said 9 before, you know, the data will be 10 useful and valuable and I don't mean to 11 understate it in any way. It is one 12 snapshot and we wouldn't predicate what 13 we do in A2/M2 certainly based on just 14 this data, although this data may 15 generate hypotheses that we might want 16 to test and focus groups sometimes 17 generate hypotheses that you want to 18 test. I will try to make this quick. I 19 will chew up all the time. 20 MR. ZACKON: Good idea. 21 MR. DONATO: Long term, two 22 trials here. We're kind of interested 23 in understanding how this works through 24 time. Maybe it is a little bit of a 25 limitation. But it is over time and 0086 1 whatever else we would do to understand 2 cross platform usage and things like 3 engagement. Other methodologies, sort 4 of longer term. Combined, I think they 5 give you a fuller picture than either 6 one separately. 7 Instrument bias. I am a little 8 worried about the instrument bias but 9 all instrument bias. People meters have 10 instrument biases. Direct observation 11 has instrument bias. Focus group has 12 instrument bias. 13 My bigger concern is, I referred 14 to this before as a magnifying glass, 15 and you get really close and you see 16 what people are really doing. But 17 almost to your point, Ceril, I'm not 18 sure, I'm not a hundred percent sure 19 that this is the magnifying glass for 20 commercial versus program. I wonder if 21 people knew that they were being watched 22 whether they would view commercials 23 differently or programs differently. I 24 don't know. That would be a question I 25 would ask. 0087 1 In some cases, I almost think I 2 might look at tuning data for that. 3 Then I had a thought, what about if 4 someone was carrying around a device 5 that listened for audio which has legal 6 problems or codes or signatures and, 7 therefore, you wouldn't have to even be 8 worried about recording. Just kind of a 9 thought of enhancing it. As we think 10 about our quantitative approach, that is 11 something that we would do probably. A 12 PDA, prompting periodically for specific 13 behaviors and -- but then also, you 14 know, we have a couple of devices ready 15 for codes and signatures portable and 16 probably signatures, not codes. You 17 know, this way would afford movie 18 theaters. The observers would carry 19 such a device which would improve maybe 20 the quality of this study. 21 Engagement, this is sort of -- 22 we're looking at various measures that 23 would actually electronically get to 24 engagement. And I would be interested 25 in seeing the data just to help me 0088 1 understand how those electronic measures 2 would work. Would it launch a program, 3 no. But it would give guidance in terms 4 of how various electronic measures of 5 the engagement would work. It would say 6 among a sample of people, this is how 7 they're using multiple media together. 8 When you're designing your 9 electronic measurements, make sure you 10 can capture those. At the end of the 11 day, you can reduce it down to a single 12 debate, single issue, and that is, you 13 know, the data have value. I think the 14 data have significant value. The 15 question is, if the data have the value 16 necessary to basically kind of take up 17 all the funds for whatever period of 18 time it is. 19 And I think what you really have 20 to think about, how you want to allocate 21 all your money. I should answer the 22 question on funding. Unfortunately, I 23 don't -- I spoke to Susan as recently as 24 last night. What I will tell you in our 25 thinking, there's no change in thinking. 0089 1 You know, we have new owners, we have to 2 present the budget. We haven't 3 presented the budget. I can't promise 4 you anything. I can say we're not 5 thinking anything differently. So I 6 would add that on the basis of that sort 7 of best information I can provide, and 8 so, that really is what it comes down. 9 Is that how you want to spend all 10 of your research funds for whatever 11 period of time, or do you want to look 12 for something which you want to kind of 13 modify the study. They're all in the 14 proposals. You may have options that 15 you didn't have as recently as 16 yesterday. 17 MR. STERNBERG: I would like to 18 make a comment. I think to some degree 19 we're having the wrong conversation 20 here. I think that our industry has 21 reached a point right now where there's 22 very little research being done. I 23 think that most of the research that is 24 being done, people don't believe in. 25 When you see a study from 0090 1 somebody that says media usage is down, 2 I don't know how much people believe 3 that. Part of the problem right now is 4 that a good portion of the industry does 5 not trust Nielsen data, does not believe 6 in the Nielsen data. Nielsen is putting 7 out data covering the industry as a 8 currency without doing any real research 9 to determine whether it is valid. 10 So to even say use this device or 11 that device or this thing that Nielsen 12 comes up, that's not the purpose of what 13 we're doing. The purpose of what we're 14 doing, one of the purposes is to say 15 there's all this other bogus or what we 16 think might be bogus research out there. 17 Is it going in the right direction or 18 wrong direction. Research is not 19 designed for precision. 20 You are never going to get a 21 precise answer from a study like this. 22 You're going to get guidance and it will 23 tell us these five studies say this, but 24 this observational technique we think is 25 better. Nielsen is saying this, but 0091 1 this is saying something else. So it 2 will lead us into what else needs to be 3 researched and possibly what else 4 Nielsen needs to be doing, which I don't 5 see happening. And I see us going 6 without a study like this, continuation 7 on this path of just worse and worse 8 mediocre research. 9 And I really believe something 10 like this is necessary to get us back, 11 at least to try to get us back on the 12 right path. 13 MR. ZACKON: I'm going to take a 14 stand for the council. We have a 15 project on nonresponse bias which I 16 think does directly take on validity. 17 So I don't know that there's no 18 research. 19 MR. DONATO: With all due 20 respect, I don't see how this gets to 21 Nielsen validity. 22 MR. POLTRACK: Steve, it also 23 comes back to the point I was asking 24 them about, to the extent that senior 25 management and the others feel that they 0092 1 want to look beyond Nielsen or have any 2 questions about Nielsen, what concerns 3 me about the study, it is so big and it 4 is so expensive and so comprehensive, 5 yet it starts with Nielsen cooperators. 6 And I think I could go, if my management 7 says how much confidence do you have in 8 the Nielsen system, I can't use this 9 study really to defend the Nielsen 10 system because it starts with Nielsen 11 cooperators. 12 I think what is missing here is 13 some attempt anyway to have a 14 sub-segment of -- I guess the Nielsen 15 people, you gave them the list of 16 cooperators. Could you also give them a 17 list of noncooperating basics to try to 18 recruit. 19 MR. DONATO: I suppose we could. 20 But my personal professional judgment is 21 I'm less worried about the difference 22 between the cooperators and the 23 non-cooperators. I don't know how 500 24 observations for two days is going to 25 tell me whether I'm measuring television 0093 1 or not. I think it is illuminating and 2 highlights issues that can be very 3 useful for the understanding of how to 4 measure multiple media, but do people 5 really believe in a household -- what 6 are you going to do, come up with a 7 rating level from the sample and compare 8 it to what we do? 9 MR. POLTRACK: I think a 10 practical situation I can give you. We 11 have, since the beginning of the work on 12 people meters and everything, one of the 13 things that we have highlighted is that 14 the current measurement system is 15 deficient at the hours of high activity 16 in the home when people return to the 17 home and are reestablishing themselves 18 in the home. 19 Now this little pilot project 20 that was the area of the greatest 21 differential in viewing where the pilot 22 said it was showing a much higher level 23 of television viewing during that four 24 to seven time period. So right there, I 25 think that's something. And I guess it 0094 1 also has to do with whether we spend all 2 this money on this general area or 3 whether we try to pick out certain areas 4 that are more relevant and focus a 5 little bit more on certain areas and not 6 try to be as global as the study is. 7 I mean, you know, they're going 8 to go out of their way to measure mobile 9 video as part of this thing. We know we 10 do research all the time. There is no 11 mobile video going on right now. How 12 much effort are we going to be making to 13 measure mobile video at this point in 14 time? 15 MR. DONATO: If you want to use 16 it for that purpose, you would be better 17 off having homes that still had people 18 meters in it. I know you have to get 19 away from the observation bias. You 20 could see how people were attracting 21 with meters. 22 Here is a legitimate question. I 23 don't mean to sound offensive. Would 24 you consider observation without meters 25 in the household to be a better 0095 1 benchmark against which to compare 2 ratings, or would you consider the 3 telephone coincidental which is 4 currently going on and been proposed to 5 the industry as an ongoing validator 6 better? And both have limitations, I 7 understand, but right now there is a 8 coincidental going on and we'll publish 9 the first results and we'll make a 10 decision as an industry to continue as 11 an ongoing validator. 12 MR. GLOECKLER: It seems like a 13 different question. This isn't designed 14 to validate. 15 MR. DONATO: I agree completely. 16 Somehow that is the avenue of debate 17 that we got into that this will validate 18 Nielsen ratings. 19 MR. LIGUORI: It seems to me that 20 as I'm listening to this, there are too 21 many expectations of the study. It is 22 exciting to listen to them and it does 23 generate other questions. But in its 24 simplest form, I thought what we were 25 looking at was how people -- how 0096 1 people's behavior is changing in terms 2 of the devices that are used. 3 I never saw this as, you know, 4 somehow a report card or, you know, 5 comparing the viewing that was done on 6 these devices to anything that you do at 7 Nielsen. This was more behavioral. And 8 it was a first step and it was to 9 benchmark, because as we go forward and 10 the devices became more pervasive 11 throughout homes, we would have a basic 12 point at which it started and to see how 13 it has evolved. 14 It just seems like there are too 15 many things being expected of this first 16 study. 17 MS. BURNS: You're right on 18 target. Read the mission statement, 19 because the sense that I'm getting here 20 is too many broad interpretations of 21 what really is a focus study. So I 22 would like everybody before we vote to 23 once again understand the mission 24 statement and what we're trying to get 25 out of this, which is not validation, 0097 1 which is not challenging Nielsen, and it 2 isn't the sole purpose, everybody in 3 this room using quantitative and 4 qualitative, we never make a decision on 5 one sole methodology. 6 If you could reread the mission 7 statement so we can put out of this and 8 not ask of it something we're not 9 intending. 10 MS. BRILL: Thank you. 11 I would like to refocus the group 12 on what our original intent was. The 13 mission of our committee and the study 14 is to improve and involve TV and video 15 audience measurement through 16 comprehensive and ongoing study of media 17 consumption. It is really to inform 18 learning for the first time to get a 19 blueprint of how consumers are 20 interacting with devices wherever they 21 may be throughout their day. We never 22 had anything of this scale before. 23 And, also, I would like to 24 reiterate what was in our RFP. Our 25 committee sought to, one, to mention the 0098 1 current composition of media focus on TV 2 and video and how it is changing and 3 will change over time in order to 4 propose the optimal form of video media 5 measurement. 6 MR. ZACKON: Ira. 7 MR. SUSSMAN: I think the purpose 8 of this is great. I think my concern is 9 the cost of doing this in light of other 10 things that might want to be done. 11 When we first talked about doing 12 this and really understand the whole 13 breadth, that's a great study. But when 14 you start looking at the advertising 15 dollars that are at risk or being played 16 in a lot of these areas, I think, you 17 know, focus on where the investments are 18 around learning, there might be -- I 19 understand why we went this way, I'm 20 just concerned about the amount of money 21 that's being pushed at it. 22 MS. BRILL: This amount of 23 money -- 24 MR. ZACKON: Actually -- 25 MS. BRILL: -- it is not a 0099 1 secret. It is something that was 2 mentioned up front when we first 3 presented our proposal to the committee 4 last September. And that is why we 5 voted to first do a pilot, to see if 6 this observational technique would be 7 viable on this grander scale. 8 MR. GLOECKLER: Is there a way to 9 split the cost over two calendar years 10 instead of having to fund this all in 11 one year? Maybe it is already planned 12 that way. Do you make it so it cuts 13 across two years? 14 MR. POLTRACK: It is proposed. 15 MR. STERNBERG: Nielsen dog eared 16 a year of funding. 17 MS. BUSLIK: I had a question. 18 It seems to me this thing grew 19 somewhere and maybe I didn't attend a 20 meeting, but I got lost somewhere. We 21 have gratification scales that I'm not 22 quite sure how we're going to use them, 23 what they really mean and not the 24 engagement. 25 Maybe we can still do the study 0100 1 but eliminate some of the nuances to it, 2 because I would rather see some of the 3 information really blown out and 4 analyzed by them and giving us some real 5 direction, rather than to ask a 6 bazillion questions. 7 So I'm looking at the gratis 8 scale and I can't imagine how I'm going 9 to use, I pity when I watch this program 10 or this type of show, I feel anger, I 11 feel love. I mean, how is that going to 12 help us understand how television or any 13 of the new technologies are used. 14 So could we maybe pull back some 15 and maybe save some money and do 16 something additionally with sample 17 that's not part of the Nielsen sample, 18 if you think that's important. 19 MR. ZACKON: Steve, in part, in 20 response to what you and Shari were 21 saying, when this was first proposed 22 there was a number as low as 1.8 million 23 in which we could get two ways. People 24 may have bought in based on that. Now 25 we don't have that lower level. It is 0101 1 not surprising that people are struck by 2 a cost increase. 3 MR. STERNBERG: I think initially 4 we mentioned that we would need two 5 years of Nielsen funding. I think the 6 1.8 kind of went away quickly. 7 MS. GALLAGHER: That was never 8 mentioned. 9 MR. WAKSHLAG: The answer is we 10 certainly should go back and express any 11 concerns that the committee has on the 12 value of these scales. I mean, it is a 13 classic way of segmenting the 14 marketplace into saying I have people 15 who use TV for X and TV for Y and those 16 type of particular scales and analyze 17 and create personality types. And 18 that's, you know, if there's little 19 value in that, I'm not sure how much 20 incremental cost that really is. 21 Collecting the data from these is 22 not the cost. The cost of collecting 23 the data from understanding is paying 24 the observers to collect the data and 25 the cost of the management. 0102 1 MS. BUSLIK: The management has 2 to be the hours that they devote to 3 analyzing this segment of the study. I 4 mean, so I'm not sure -- fine, I 5 understand segmentation, but what can we 6 get out of that. How can we apply that 7 to the future. 8 MR. POLTRACK: We're doing this 9 research now. Anytime you do this 10 research, you need a sample of thousands 11 to apply that kind of study, that kind 12 of study. It is not the kind of 13 segmentation that really works with a 14 small sample like this. 15 MR. WAKSHLAG: I agree, if you do 16 these types of analysis, you need more 17 of a sample. 18 MR. BROOKS: I think everybody 19 has focused on, people in the room, and 20 gotten a breakthrough of understanding 21 of something in their career. I suggest 22 to the group that you step back a little 23 bit. This is an illusionary study. It 24 is not a syndicated study done every 25 year or every month. It is just too 0103 1 expensive a methodology. 2 But once a decade, once there is 3 a major change in what the media map is, 4 trained observers watching a broad range 5 of people actually using these media and 6 how those media intersect with each 7 other seems to me a basic study that we, 8 as users, we need to understand that and 9 we can go down self-reporting and go 10 down hunting people meters, go down 11 whether the set is on or not, but on an 12 occasion like this, this started from 13 two words, media map. 14 What is the media map in 2008? I 15 think it is probably the only 16 opportunity any of us are going to have 17 in quite a while to step back and get 18 that kind of basic understanding of 19 what's going on. I think it has to 20 inform you because you understand better 21 how your clients, how your consumers are 22 using different media and how those 23 media interact with each other. 24 MS. RUSH: I think every day in 25 our home what consumers are doing with 0104 1 media and what they're doing, I think 2 the study would be very important in 3 terms of understanding those behaviors 4 because these questions are impossible 5 to answer at this point. 6 MR. DONATO: I almost think the 7 benefit goes the other way. In theory 8 what is unique is the observation and 9 that's the thing you're never going to 10 have among 10,000 persons or households. 11 So you have this unique data. And now 12 regarding the gratifications questions 13 and the ancillary data that you collect, 14 you should be intending, be able to 15 predict what is unique in this study. 16 So there are certain things that 17 you can ask that somehow are predictive 18 of or correlate with observations so 19 that then you could go outside of 20 observation and ask that of much larger 21 groups and kind of trust your 22 regressions, trust your predictions and, 23 therefore, say, you know, that's the way 24 a larger quantitative -- I'm not 25 articulating very well. Try to predict 0105 1 through questions what you see in 2 observation. Then you get a much larger 3 sample. You ask the same questions and 4 it is if you have the observation for 5 the larger sample. 6 MS. RUSH: I understand, got to 7 save the last half hour, Paul. 8 MR. WAKSHLAG: The reservation I 9 have about the study in general is, and 10 I'm going to talk about it in some 11 depth, is that it will give me a media 12 effort today. It doesn't give me a 13 media map of two or three years from 14 now. I will observe what people do in 15 their households today and then go back 16 into the marketplace eight months later 17 and, you know, if all of the sudden 18 there is a huge increase or change in 19 the kind of technologies people have 20 between the first of the year and 21 October, then I might be able to project 22 something into the future. Or I can 23 look back and talk about so and so 24 people had DVRs and they bought DVRs or 25 had DVRs for a similar length of time. 0106 1 The one reason I like doing 2 January and October, I'm looking for 3 robust findings. I like the results to 4 be whatever I observe in January to also 5 hold true when the television season 6 begins and baseball is on in October. 7 And if I find the same things in this 8 two months, I have much more confidence 9 in a solution that covers multiple 10 seasons. So in one way I thought maybe 11 we should wait January to January. 12 No, I'd rather test this thing in 13 various situations. But the other 14 thing, you know, I'm going to talk about 15 a study that Don knows and some of the 16 others have used. We're not exactly 17 sure exactly how we're going to design 18 wave two of this study. We can go in 19 and do the exact same thing we did in 20 wave one and observe these people again, 21 or we could adopt another study which 22 I'm familiar with and which I've 23 discussed with the other members of the 24 committee. Don is familiar with it 25 because his company sponsored it. 0107 1 Time Warner, Pepsi, P&C, funder 2 of the study that these guys -- media 3 acceleration project. What they did was 4 a small sample of 50 people. And we 5 gave them a credit card. A credit card 6 allowed them to go to Best Buy and buy 7 from a catalog of products people who 8 were designing the study had gone 9 through, and we would subsidize the cost 10 of that purchase to the tune of 50 cents 11 on the dollar. So they could go to Best 12 Buy and they could buy a wireless P.C., 13 you know, wireless modem, et cetera, 14 et cetera, which they might not buy for 15 a couple of years because it cost too 16 much. Or a video iPod. The media 17 acceleration project is because we try 18 to reduce the cost barrier for new 19 technologies to the kind of things. 20 MR. ZACKON: I'm going to step 21 in. If you could bottom line. 22 MR. WAKSHLAG: There were some 23 observations that were useful. They are 24 profound observations. The people who 25 bought iPods, they didn't spend time on 0108 1 video, even though they bought video 2 iPods. What they did do was spent a lot 3 less radio time. That is important to 4 my business and your business to know. 5 In fact, I just read this morning 6 CBS radio concerned about the 7 demographics of radio because he said in 8 this article that I read that young 9 people are starting to migrate to these 10 other platforms and he's comfortable 11 changing the CBS affiliate in this 12 market to a 35 to 54 radio station. 13 So that happened. People who 14 bought games with this money spent a lot 15 of times with games and that came at the 16 expense of a lot of other stuff. Almost 17 everything. People who bought wireless 18 laptops spent less time on the internet 19 than they did before. This was a three 20 wave site. Why did they spend less 21 time? Because they didn't have to go 22 off to a separate room where they had to 23 turn the thing on and sit there and 24 spend time with it. 25 Now, I'm not saying these rules 0109 1 are definitive, but it does give us an 2 option in terms of looking at the data 3 and seeing which way to go with wave 4 two. I believe the study -- and if we 5 don't want to do it here, I'll find 6 another way to do it. 7 MR. ZACKON: Are we being asked 8 to vote today? 9 MR. WAKSHLAG: No. 10 MR. LIGUORI: One of the things 11 you should consider is that there is 12 likely to be a shift, a pretty big 13 shift, in my opinion, come February 17, 14 2009, because everything that's portable 15 is digital. And for those -- and to 16 assume those that don't have that in 17 their home are the downtrod and the poor 18 is missing a segment that just doesn't 19 have everything connected that may chose 20 to spend their money on a portable 21 device instead of having another set. 22 So there is going to be a major 23 change. Now do we say wait, or do we 24 say let's benchmark what's happening now 25 and then see what happens then? 0110 1 MR. ZACKON: So there's a lot we 2 could continue to discuss it. We could 3 spend the rest of the day discussing. 4 It is up to the group how to do it. 5 Shari, Steve, do you have a 6 particular motion you would like the 7 committee to move on hearing based on 8 what you're hearing? 9 MS. BRILL: I just want to steer 10 the committee back to what the council, 11 as to what our vote is based upon today. 12 Is there any more discussion or should I 13 say that? 14 MR. ZACKON: Why don't you say 15 what we're here to vote for. 16 MS. BRILL: Two-stage 17 observational study with the potential 18 to review after our stage one. A sample 19 ranging between 350 through 500 people 20 per wave. The cost not to exceed 3-1/2 21 million, contingent upon Nielsen funding 22 a third year. And that's the top level. 23 The next phase would begin 24 January 2008 with a second phase 25 beginning October. 0111 1 MS. SHAGRIN: A second specific 2 question to that statement, is for some 3 reason Nielsen doesn't fund a third 4 year, what happens? It says contingent 5 upon. Does that mean we don't do the 6 second phase? 7 MS. BURNS: The second phase is 8 voted on. After we see the results of 9 the first, we'll decide. 10 MR. WAKSHLAG: It is a two-phase 11 study. We'll have to talk about how 12 small the sample should be. 13 MS. SHAGRIN: I'm looking, can we 14 cut costs by limiting it to three 15 markets? You know, does that bring the 16 cost down to 2.8? 17 MR. ZACKON: We currently have, 18 from my calculation, this is in your 19 packet, $3,294,000 is available. That 20 doesn't assume the universe assessments 21 going to be funded, the marketplace 22 practice is going to be funded. That's 23 just based on what is approved. 3.3 24 million available of Nielsen's 25 contribution. 0112 1 MR. SHIMMEL: I think it is 2 important in terms of what you are 3 voting on today, it is 350 or 500 X 4 Nielsen sample. I would think if a 5 decision was to have a hundred high-tech 6 people outside the NMR sample, the cost 7 would be different. 8 MR. IVIE: I thought that wasn't 9 in the plan. 10 MR. WAKSHLAG: That's not in the 11 plan. 12 MS. BURNS: They anticipated a 13 higher cooperation rate. 14 MR. SHIMMEL: I understand that. 15 MR. WAKSHLAG: It is not the 16 committee's recommendation. 17 MR. SUSSMAN: The proposal is a 18 two-wave plan. 19 MR. ZACKON: There is a problem 20 if the second phase doesn't move 21 forward, that is 300,000. 22 MR. STERNBERG: There is a 23 termination fee. 24 MS. SHAGRIN: But is the 3.5 25 divided by 2 plus 300 the amount we are 0113 1 committing to the first phase? 2 MS. BURNS: No. 3 MR. STERNBERG: 3.5 is committed. 4 If Nielsen funds the next year, that's 5 for the full two phases. 6 MR. ZACKON: The committee 7 requests two-way funding. 8 MS. SHAGRIN: I understand. I'm 9 just trying to say if it doesn't happen, 10 what is the exposure, how much of the 11 money that is already allocated would we 12 lose. 13 MR. BROOKS: Remember, this is a 14 ceiling amount. The committee wanted to 15 be -- to get the maximum amount. The 16 committee is upset about the cost, too, 17 and we had a fiery meeting with the 18 partners about that, as a matter of 19 fact. 20 MS. BURNS: And they're still 21 sharpening their pencils. 22 MR. WAKSHLAG: I was going to ask 23 George if this is possible, that's the 24 maximum amount the committee is 25 requesting to be approved. Obviously, 0114 1 if Nielsen does not come forward with a 2 third year of funding, the project would 3 have to come in at a lower amount. 4 MR. STERNBERG: I think one 5 question is also if Nielsen does not, 6 are we now committing the 3.2, 99, 7 whatever it is available to this. 8 MR. ZACKON: Everything remaining 9 is what you're saying. 10 MS. BUSLIK: I would like to 11 suggest that the committee go back and 12 discuss this again with the partners, 13 asking them for some alternatives that 14 do not eat up the 3.5, which may scale 15 it back, and give you the option of 16 doing first wave to see what comes out 17 adding additional things. 18 MR. WAKSHLAG: If we chose not to 19 do a two-wave study, the cost of the 20 first wave is -- 21 MR. STERNBERG: 2.6 million. 22 MS. BUSLIK: I would like them to 23 come back and go back to them even for 24 the first wave and have them repencil 25 and re-erase some numbers, maybe scaling 0115 1 back to some degree. 2 MR. ZACKON: There is a motion on 3 the floor to fund 3.5 million contingent 4 on Nielsen funding a third year. 5 MS. BRILL: Up to 3.5. 6 MR. BROOKS: Maximum. 7 MS. BRILL: We're still 8 negotiating with them. 9 MR. POLTRACK: Minimum is 2.6. 10 MR. ZACKON: 2.6 gets you 400 11 people for one event. 12 MR. POLTRACK: If the study goes 13 forward and we do phase one only, we 14 have to pay 2.6. 15 MR. KEILTY: Plus the 300,000. 16 MR. ZACKON: That includes it. 17 MR. LIGUORI: Does spending the 18 3.5 preclude other studies? 19 MR. ZACKON: Yes. 20 MS. GALLAGHER: That precludes 21 all of the studies. 22 MR. WAKSHLAG: Wait. The 3.5 23 assumes another year of funding from 24 Nielsen of an additional 2.5. 25 MR. STERNBERG: I would like to 0116 1 make another motion. 2 MR. WAKSHLAG: There would be 3 another $2 million if something else -- 4 MR. ZACKON: Jack, one second. 5 He would like to revise his motion. 6 MR. STERNBERG: I would like to 7 propose that we're voting on funding 3.5 8 million contingent on Nielsen funding a 9 third year. If Nielsen comes back and 10 does not fund the third year, we revote 11 so that you are not committed to use all 12 the funds that are available to the 13 entire committee at this point. 14 Our committee has discussed 15 extensively whether we wanted to do a 16 two or one-phase study. We decided we 17 didn't want to propose a one-phase 18 study. I would much rather at that 19 point vote to fund 2.5 if another 1.5 20 comes into the till. If not, we take a 21 revote. 22 MS. SHAGRIN: When do you think 23 there will be a definitive answer? 24 MR. DONATO: I'm hoping in a 25 month. Our budget cycle got moved back. 0117 1 I can't guarantee. If it is not a 2 month, it won't be two months. 3 MS. SHAGRIN: What is the latest 4 we can start? That means that we really 5 can't start the project. 6 MR. STERNBERG: We are going in 7 the field in January. There is time 8 before we proceed. 9 MS. GALLAGHER: They have to hire 10 observers and stuff. I assume they have 11 a certain amount of lead time that 12 they're going to require. 13 MR. BROOKS: They originally 14 proposed this fall, and because of the 15 timing, this delayed it to January. 16 MR. ZACKON: So a no vote might 17 not reflect on the value of the study. 18 It might simply say that we're not ready 19 to decide that at this meeting. Is that 20 correct? 21 MR. LIGUORI: Yes. 22 MS. BUSLIK: That's not his 23 proposal. 24 MR. ZACKON: What I'm saying to 25 the people, are you still open to this 0118 1 but not ready at this meeting to commit 2 that no is an appropriate vote. If you 3 are open to commit, that's yes. 4 MS. BURNS: No, I would like, no, 5 don't even bother working with them to 6 refine the cost or, no, we're not 7 prepared to vote today are two different 8 things. 9 MR. ZACKON: The latter is what 10 we're saying. 11 MR. WAKSHLAG: There is a bad 12 idea for a study versus or we should do 13 it contingent on Nielsen's coming 14 forward with 2-1/2 million. 15 MS. GALLAGHER: Or it is not a 16 bad idea for a study but the cost is 17 such that people might want to consider 18 some other alternatives. 19 MR. ZACKON: Yes vote with 20 Nielsen putting up the money commits to 21 this study, those two events happening, 22 the study continues. 23 MR. WAKSHLAG: Plus the costs 24 come. I'm going to ask George and 25 Nielsen actually, because none of us are 0119 1 financial experts. We don't know what 2 per diem costs for data collection are, 3 but we look at it on an occasional basis 4 that Y & R, can we recruit Y & R to 5 review in detail. 6 MR. ZACKON: E & Y. Y & R has 7 been on the study. 8 MR. WAKSHLAG: Can we get E & Y 9 to agree and review and make 10 recommendations on the costs of this 11 study and also the Nielsen, whoever does 12 this kind of costing at Nielsen, so we 13 know -- we want to make sure that we're 14 getting the price at the right price. 15 MR. ZACKON: If I can be clear. 16 What would the mechanism be to agree to 17 price? How does that happen if this 18 group doesn't agree? Is there a 19 subsequent vote on the final price or 20 the chairman looks at it and says fair 21 price? 22 MR. IVIE: If you can get a 23 hundred thousand dollars or 300,000 out 24 of the price, I'm not sure that changes 25 the picture. I think we're being overly 0120 1 complex. As somebody who votes all the 2 time, stuff very difficult to 3 accomplish, I can tell you we're being 4 overly complex. 5 I think the motion that has been 6 made, we either approve the study 7 contingent upon Nielsen funding, and if 8 they don't fund, we come back and talk 9 again, or you don't like the study. So 10 you have those two options. To me, that 11 makes sense. 12 MS. GALLAGHER: I don't like the 13 way that's worded. It is not like it is 14 okay, it is a good study, we fund if 15 Nielsen comes up with the money or we 16 don't like the study. It is not 17 necessarily that someone doesn't like 18 the study. It might be that they don't 19 think there is $3 million of value in 20 there. It might not mean that. It 21 might mean the study doesn't have 22 $3 million worth of value. 23 MR. ZACKON: Somebody has to say 24 this has been negotiated fairly and I 25 want to know how that process is clear, 0121 1 otherwise, we need to come back to this 2 group for a vote, which I would like to 3 avoid. 4 How would that be determined, 5 just a say so the committee says. The 6 steering committee gets involved. 7 MR. IVIE: That's what I would 8 do. Jack was saying Nielsen would get 9 involved. I could bring in some people 10