288 00:46:54.060 --> 00:46:55.860 Campbell Collaboration: Is share my screen. 289 00:46:59.790 --> 00:47:06.660 Campbell Collaboration: Hopefully you can now see the screen. I can see it. I see it. So that may not be a good 290 00:47:11.790 --> 00:47:12.660 Susie Burdett: We can see it. 291 00:47:13.560 --> 00:47:14.370 Yeah, thank you. 292 00:47:16.050 --> 00:47:26.670 Campbell Collaboration: So I'm going to talk a little bit of a mid level theory, not too much because most what I plan to say Nancy's. TOUCHSTONE I have some exotic some more examples. 293 00:47:27.720 --> 00:47:33.960 Campbell Collaboration: Which you to attend my presentation. Back in April is the development of those slides, and I won't spend too much time on it. 294 00:47:35.400 --> 00:47:36.210 Campbell Collaboration: Then I'm going to 295 00:47:37.560 --> 00:47:54.660 Campbell Collaboration: focus of my talk is on an analysis I did of how these that is using mot are proposing to use it. And so first I asked Otter cosplay takes a very pretty name or two. So there are four basic approaches bottom up top down Initiative, a mathematical mathematical modeling. 296 00:47:56.130 --> 00:48:01.590 Campbell Collaboration: The topology to be more stylized when you actually see in practice by finding helpful to think from that way. 297 00:48:02.160 --> 00:48:12.120 Campbell Collaboration: And I might ask what the relationship is punishment penalty in the study how they're planning to use mo tunes study, are they using the studies or bad naughty. Are there continues to the 298 00:48:12.570 --> 00:48:21.750 Campbell Collaboration: The evidence to test, there is or they're going to use the theory to test interventions or they're going to use it. The theory to predict impact on the new system. Nancy talked about 299 00:48:22.470 --> 00:48:30.180 Campbell Collaboration: You can of course also use mid level therapy program design that's not done as well. I can see the programs where we have satisfied were founding 300 00:48:31.920 --> 00:48:35.310 Campbell Collaboration: And then find out where to go over and over to you have the rest of the workshop today, tomorrow. 301 00:48:36.660 --> 00:48:44.730 Campbell Collaboration: And so just on what middle level theory is is developed from Robert Merton sociologist working back in the 50s and 60s. 302 00:48:45.090 --> 00:48:57.150 Campbell Collaboration: Wrote about mid level theory, yes. Use them to a middle range theory. But now, it won't allow us to say that you said about being one thing. So we use the term mid level or middle level theory instead 303 00:48:58.740 --> 00:49:12.000 Campbell Collaboration: And what what Merton said was natural distressed merchant wanted something between brand theory, which was two grand to actually be of any practical value. She said Marxism was an example. 304 00:49:13.320 --> 00:49:26.910 Campbell Collaboration: And sort of just data observation. And so the idea is, it should be as my as my third bullet. So since between a theoretical empiricism, which may be some randomized control trials can be accused of it computer system. 305 00:49:27.330 --> 00:49:36.300 Campbell Collaboration: And and grand theory which has no no practical application. And so it's based on having clear, defined self nominate 306 00:49:37.170 --> 00:49:49.770 Campbell Collaboration: Like say social groups rather than much more abstract entity like society and so should be based on data which is testable. So that's the key thing come up with propositions that are testable using data. 307 00:49:50.520 --> 00:49:59.730 Campbell Collaboration: And there's been increased interest in middle theory. I'm settled settled Patil I think that movement, but if I have spoken about in other places as well. 308 00:50:00.000 --> 00:50:08.730 Campbell Collaboration: And for us, and I think many others, the interest has come because the focus on transfer ability. So it's always been a question in the back to version literature. 309 00:50:09.030 --> 00:50:22.980 Campbell Collaboration: Okay, can you transfer. How do you use funds from one study to talk about other settings, our contacts our programs on places and middle level theory is meant to be a framework to help think about 310 00:50:24.450 --> 00:50:31.980 Campbell Collaboration: So what I'm going to do in the first part of my talk is give some examples of of applying this approach. 311 00:50:33.330 --> 00:50:39.450 Campbell Collaboration: Which hopefully the, the idea is to sort of stimulate you to thinking about how you're using yourselves. So 312 00:50:39.780 --> 00:50:48.390 Campbell Collaboration: Here's a middle, middle level principle and, as Nancy said, and I said the replies middle can pay for, like, on a scale up I hired him quite low. 313 00:50:49.380 --> 00:50:58.230 Campbell Collaboration: So this is providing people information about the benefits of a certain heart of best behavior or the house and not adopting the behavior, increase the likelihood of behavior being adopted. 314 00:50:58.590 --> 00:51:06.180 Campbell Collaboration: That's an estimate. And I said, found sound biblical principle of which they would be specific example strip theories of change. 315 00:51:06.390 --> 00:51:17.040 Campbell Collaboration: Around family planning, I caught extension website, you can find an anti smoking all the things I tell you to do something or not do something that benefits arms that will follow from adopting or not adopting that behavior. 316 00:51:19.320 --> 00:51:32.070 Campbell Collaboration: As up. Alright, so what did. What are the assumptions underlying this this causal principle this course or principles physical principle on the line calls or principal assumptions that need to hold true 317 00:51:32.790 --> 00:51:39.990 Campbell Collaboration: In order for this course or principal operate and Nancy classify the assumptions in different ways. Now come back to that and show an example of that. 318 00:51:40.800 --> 00:51:49.860 Campbell Collaboration: In a moment, so I got for example for assumptions here that could that LinkedIn is causing principle, being able to operate. 319 00:51:50.250 --> 00:51:55.260 Campbell Collaboration: So the first is that the behavior. You're telling people to adopt is not already been adopted. 320 00:51:55.620 --> 00:52:08.610 Campbell Collaboration: So one of the first theory based evaluations carried out by the World Bank and summer we did that cost extension service in Kenya found out that the extension works for telling hours to adopt practices actually already doing. 321 00:52:09.690 --> 00:52:14.190 Campbell Collaboration: And so there could be no change in factors because the practice or there was two things I've been told to do. 322 00:52:14.640 --> 00:52:22.740 Campbell Collaboration: So the middle level principle in order is order in order for behavior change communication have an effect desired behavior is not already adopted in the target population. 323 00:52:23.430 --> 00:52:32.490 Campbell Collaboration: Which seemed to be a necessary but not sufficient condition for the cause or principle to operate under, under some circumstances will be continuous function or not going to go into that. 324 00:52:34.230 --> 00:52:49.710 Campbell Collaboration: Just to explore how you might express that pose or principle, you could choose explicit formally formally but action exit time t may lead to outcome wine time t plus one, even though the 325 00:52:50.070 --> 00:52:53.580 Campbell Collaboration: Exit time t minus one was zero debt in don't have that behavior. 326 00:52:54.150 --> 00:52:59.820 Campbell Collaboration: Or you could express it more simply as tell people things they didn't already know. Okay, so, as Nancy said 327 00:53:00.150 --> 00:53:07.920 Campbell Collaboration: It was open source. Often they seem obvious army mean evaluator for rather many years now, and often immigration policy, just 328 00:53:08.220 --> 00:53:15.000 Campbell Collaboration: Why didn't I think of that, it's always tell people things they don't already know the sound obvious, but to echo the point that 329 00:53:15.510 --> 00:53:23.310 Campbell Collaboration: The questions. Sometimes it is well worth putting calls or stating the obvious because people do things that contradict the office. 330 00:53:23.760 --> 00:53:29.790 Campbell Collaboration: The second underlying assumptions is possible to adopt the intervention interpreted through something. Is it possible 331 00:53:30.180 --> 00:53:37.230 Campbell Collaboration: To an impact environment training farmers in the regatta culture had no impact because we irrigation infrastructure hadn't yet been built 332 00:53:37.830 --> 00:53:41.160 Campbell Collaboration: So they couldn't adopt a new agent. Perhaps it was there was no regression 333 00:53:41.790 --> 00:53:49.890 Campbell Collaboration: So mid level principle there is if you tell people perform action X and you compare your inputs production next must be available and accessible. 334 00:53:50.130 --> 00:53:57.420 Campbell Collaboration: And of course applies a wide range of setting I caught extension obvious one, but that's a common constraint, but it could be family planning, it can be road safety. 335 00:53:57.660 --> 00:54:13.620 Campbell Collaboration: All the different things I mentioned that say an often reason why cause a principle that operate is the things people need to do the things I do aren't available accessible to them. And again, that's a necessary but not sufficient condition and again it could possibly be continuous. 336 00:54:15.060 --> 00:54:19.410 Campbell Collaboration: I want, I want to address this next to him any more any more detail. 337 00:54:20.640 --> 00:54:23.790 Campbell Collaboration: The assumption three is it should be beneficial to adopt intervention. 338 00:54:24.120 --> 00:54:35.370 Campbell Collaboration: And I do strongly believe that most program managers overstate the benefits of the intervention to the intent of beneficiaries and underestimate or ignore toast with the costs of participation. 339 00:54:35.940 --> 00:54:43.920 Campbell Collaboration: And just in terms of production transaction costs, but the example I've got a study I did in entrepreneurship self help groups and 340 00:54:44.310 --> 00:54:49.350 Campbell Collaboration: The be productive self help group had been given a nursery sizzling project. 341 00:54:49.860 --> 00:54:57.420 Campbell Collaboration: By the NGO running self help group project in this area of part of entrepreneurship district which was inherently last maker. 342 00:54:57.810 --> 00:55:06.420 Campbell Collaboration: Just from the financial structure of the program, the why of setup wouldn't have any loss making so it's women ended up in depth as a result of the program, they did do it, but as 343 00:55:06.630 --> 00:55:12.960 Campbell Collaboration: I said, Will you do it again and looked at me like ourselves a video because they they would do the gangs. They do it was it was last making 344 00:55:13.380 --> 00:55:17.400 Campbell Collaboration: And so tell people to do exactly it's beneficial for them to do. 345 00:55:18.360 --> 00:55:21.900 Campbell Collaboration: And the final underlying assumption is it communication. 346 00:55:22.200 --> 00:55:32.280 Campbell Collaboration: And the thing is is conducting appropriate way through appropriate channels or media. So in a review, we did a film a film schools, we've found out that 347 00:55:32.490 --> 00:55:42.510 Campbell Collaboration: Very often community workers can speak the local language know 2009 years across Africa or hundred Nigeria alone his inability to speak local language is a genuine issue. 348 00:55:42.840 --> 00:55:54.450 Campbell Collaboration: But of course it seriously undermines potential impact of the program. If your work is come speak the language of the people that speaker and most African speak more than one language that you still have the problem that you are speaking your language by 349 00:55:55.230 --> 00:56:00.840 Campbell Collaboration: The Greens to talk to, you know, most comfortable wave or even create some barriers by the fact you're not speaking 350 00:56:04.140 --> 00:56:20.220 Campbell Collaboration: So there are examples of assumptions simple cause of principle. Don't people think thing is good or bad. And now transit behavior actually little and assumptions online Eccles principal and I just given for the examples where they've they've not held 351 00:56:21.750 --> 00:56:32.400 Campbell Collaboration: So I we talked about transfer ability to settle program was called in house or transfer ability middle level theory says an example we have project level finding 352 00:56:32.730 --> 00:56:41.730 Campbell Collaboration: And we convert that middle level principle which we then use to cut that down to a transferable to a lesson for another program and thinking about program design. 353 00:56:42.120 --> 00:56:51.960 Campbell Collaboration: So it's the first assumption or something I mentioned the finding from the Kenya cost extension. Is it that was being used the relative to that in Kenya. 354 00:56:52.560 --> 00:57:02.910 Campbell Collaboration: Was a farm was already using techniques been promoted. So the middle of finding their way, they will principle is to promote behaviors which are not already present in the target population. 355 00:57:03.330 --> 00:57:16.230 Campbell Collaboration: Tell people stuff that no do not so transferring that back down to a program. Most of us model contraceptives, then you'd be targeting your messaging around model contraceptives to population group of local preference. 356 00:57:17.400 --> 00:57:22.650 Campbell Collaboration: Research carried out to identify the urban population groups. 357 00:57:24.060 --> 00:57:30.540 Campbell Collaboration: to unpack that a bit more. Here's an example of a causal chain set of causal principles. 358 00:57:32.040 --> 00:57:45.990 Campbell Collaboration: About providing model contraceptives to reduce unwanted pregnancies and abortions. So provide information put bottom contraceptives, which leads to increase northern Vermont and contraceptives, which leads to increase use model contraceptives produced 359 00:57:46.380 --> 00:57:59.790 Campbell Collaboration: Pregnancies which which is the abortions. And so a lot of assumptions on the line that goes to China is very simple particle time as are any case. And I'm not going to go through all of these, but these are examples, taken from 360 00:58:00.780 --> 00:58:12.000 Campbell Collaboration: Nancy's presentation of having an overall theory, which is funding federal beneficial Baptist Church knowledge and measurable theory which related specifically to contraception, abortion. 361 00:58:12.780 --> 00:58:19.200 Campbell Collaboration: And then we took a nameless type example contraceptives without accessible to get to them. Step two to three. 362 00:58:19.800 --> 00:58:30.150 Campbell Collaboration: Steps on the bottom of the slide here, you can say it the realness of things like partners housing use of contraception, or having contraceptives don't work because the efficacy of drugs in 363 00:58:30.450 --> 00:58:36.630 Campbell Collaboration: Developing countries is serious problem that does include contraceptives the safeguards are ensuring that 364 00:58:38.100 --> 00:58:43.980 Campbell Collaboration: All decision makers in the in the process like mug with might include mother in law's mannequin mothers multitude father the husbands. 365 00:58:44.280 --> 00:58:56.370 Campbell Collaboration: are engaged in the communications that you do and what's the range of applications, their settings with north of contracts is low social norms and mitigate against the use of contraception as a women have a right basically right to choose. 366 00:58:56.970 --> 00:59:15.120 Campbell Collaboration: To do what they do with the objective help and contraceptives are available. So here's, here's a whole set of principles at the top are the assumptions that underlie them for that calls with trying to operate so seeking to illustrate here in the examples is I practically oceans of 367 00:59:16.410 --> 00:59:27.240 Campbell Collaboration: Content what NASA carbon away from doing this, you're serious about an evaluation during, during the evaluation evaluating program, you can't get away. Well, thinking, thinking in this way. 368 00:59:27.960 --> 00:59:35.940 Campbell Collaboration: So you could ask what is the value added of using Moodle at mid level theory and leading us to study it does. I'm using the word 369 00:59:36.450 --> 00:59:42.690 Campbell Collaboration: Here so sorry principles, it does visit tension might have exclude attention to the mid level causal principles. 370 00:59:43.410 --> 00:59:50.460 Campbell Collaboration: Just intervention effects are not talking about Teslas intervention, we're talking about. So this principle that's the, that's the 371 00:59:51.180 --> 01:00:01.110 Campbell Collaboration: Private mode, you want to come to with the study. So economists often do that in terms of, okay, here's a model I'm going to test that iraq water velocities have been athletic 372 01:00:02.610 --> 01:00:06.480 Campbell Collaboration: And so it's it's just a voice is racism that you might otherwise have. 373 01:00:06.990 --> 01:00:16.650 Campbell Collaboration: It what, to me it does is layout water. The whole set of assumptions that need to be tested. Because what I often say, are what we call the black box. 374 01:00:16.890 --> 01:00:25.950 Campbell Collaboration: Evaluations that look at the effect reinvention down, come down. Really, especially young black, the black box to understand why rather than just what 375 01:00:26.280 --> 01:00:34.170 Campbell Collaboration: And mid level like anything of change approached me level three helps to do that. I think it helps you do it in a more generalized why 376 01:00:34.470 --> 01:00:40.200 Campbell Collaboration: That helps with the final point of reporting in front facing or finding the wind like some more transferable. 377 01:00:40.860 --> 01:00:48.060 Campbell Collaboration: But I think that's something we'll start is, I think, bit of a really comes out in five years, I think any reviews don't use middle theories 378 01:00:48.510 --> 01:01:00.000 Campbell Collaboration: Not, not doing it right because men are there is clearly a framework allows you to structure your research question for the review around a set of calls or principles and 379 01:01:01.020 --> 01:01:04.800 Campbell Collaboration: Analysis provides motivation and as the barriers and facilitators. 380 01:01:05.460 --> 01:01:20.550 Campbell Collaboration: dividers and safeguards and you get to report something at the level mid level goals or principles that makes it makes it very clear to what the range of context around your settings are in which you think the findings are are applicable. 381 01:01:21.930 --> 01:01:30.390 Campbell Collaboration: So let's move on now to the main part of the presentation, which is about where big number theory in sort of understand is, is coming from. 382 01:01:30.870 --> 01:01:41.730 Campbell Collaboration: So I've classified this in four ways of every study that we've got here know presented hit it represented here today. I put in one of these boxes and you may object and that's fine. 383 01:01:42.540 --> 01:01:48.420 Campbell Collaboration: So nothing is stylized and and some people all in more than two bit of bit of best bit of that. 384 01:01:49.380 --> 01:02:00.030 Campbell Collaboration: But it does seem to some extent, we've got approaches and I'm going to explain the mobile in a bit more detail of approaches to advocate across there is a change. So go always do is a judge, I'm going to make one 385 01:02:01.200 --> 01:02:10.290 Campbell Collaboration: generalize the real change from aggregating across them. That's bottom up top down is here's my theory of change. Now I'm going to go out and test it. 386 01:02:11.640 --> 01:02:20.970 Campbell Collaboration: And then you're interested product resource goes between which goes between the two, which I think in the end, everyone will do so in a practical sense and then even mathematical modeling. 387 01:02:21.660 --> 01:02:25.410 Campbell Collaboration: Modeling is actually a form of top down. If you just start with your model. 388 01:02:26.280 --> 01:02:33.360 Campbell Collaboration: But it says is a specific kind of a specific type and we've got quite a lot of them. So I pulled math as a separate category. 389 01:02:33.690 --> 01:02:48.960 Campbell Collaboration: And and it does have the characteristic of being more about predicting impact was was once you've got the model you can principal do that and so that starts with got under, under model have more that orientation and do the, the other top down status. 390 01:02:50.610 --> 01:02:58.410 Campbell Collaboration: And so here's the the bottom so example of the bottom up approach rather than or tea. So you might have a school based 391 01:02:58.770 --> 01:03:10.830 Campbell Collaboration: Education intervention, you might have a graduate of measuring campaigns. Most of sex might have roadside pastor campaign project safe sex all of those basically have the causal principle body measure improves knowledge. 392 01:03:11.520 --> 01:03:21.450 Campbell Collaboration: And we're doing that because we're seven calls or principle that changing knowledge change in practice. So we'll visit bottom up approach, we've got some different interventions doing different 393 01:03:22.200 --> 01:03:28.380 Campbell Collaboration: Activities are all based on common cause of principle, providing information proving knowledge and then proving knowledge. 394 01:03:28.980 --> 01:03:38.940 Campbell Collaboration: And knowledge transfer practice. I do think these two causal principles are amongst contenders for the wild weakest link in the causal chain as I really don't think they work. 395 01:03:40.860 --> 01:03:47.640 Campbell Collaboration: Yep, good luck, we haven't mentioned based on these these causal link. So let's go ahead and test them I'm 396 01:03:48.780 --> 01:04:01.560 Campbell Collaboration: An example of bottom up, I think, is the study, we have on scattering subs telecom company for health because they're looking at individual projects and then as it says, going beyond individual projects to think about 397 01:04:02.820 --> 01:04:16.740 Campbell Collaboration: How and why they're integrated the political policy making processes and what the effect of that is so it's mid level theory comes up, not about the product to help project, but about this accountability mechanisms and how they fit into the local political economy. 398 01:04:17.190 --> 01:04:21.690 Campbell Collaboration: And so that's coming from looking at the individuals into coming to their have not 399 01:04:22.860 --> 01:04:28.770 Campbell Collaboration: I do have a view on how to do both approaches comes only when using stakeholder engagement, not only evidence 400 01:04:29.130 --> 01:04:39.870 Campbell Collaboration: And thinking about your bottom approaches and stakeholder engagement doesn't mean just researchers, it means also practitioners beneficiaries and so on. But the picture I have his ma 401 01:04:40.650 --> 01:04:51.900 Campbell Collaboration: exercise we did about the frame of reference map on youth employment. This is the research to but I wouldn't normally do just research team. It's just getting the idea just type in interactive stakeholder experience. 402 01:04:53.010 --> 01:05:00.660 Campbell Collaboration: So if you're going to do something like this, you maybe want to stop some product level, there is a change in a common format, maybe not an either too structured 403 01:05:00.930 --> 01:05:09.360 Campbell Collaboration: But either way, I'll try to, first of all, smaller, smaller team of 610 cycles from different backgrounds and then members of the research team. 404 01:05:10.590 --> 01:05:17.280 Campbell Collaboration: So you're presenting the theories of change discuss calm mechanisms at each stage of the call will try and try and 405 01:05:17.700 --> 01:05:28.530 Campbell Collaboration: Work those middle principles that cut across the theories of change and do the sign of 1000 facilitators, so you're working from product levels, there's a change to a minimum of 30 to change, working with a said 406 01:05:29.010 --> 01:05:32.670 Campbell Collaboration: The research team itself but also local researchers, practitioners 407 01:05:33.060 --> 01:05:39.000 Campbell Collaboration: program staff and intended beneficiary is going to be in that in that group do all in separate exercises. 408 01:05:39.270 --> 01:05:45.480 Campbell Collaboration: And then do what I call a mid level theory walk which an idea I stole from Sherman Robinson, what it was. If pre doing 409 01:05:45.780 --> 01:05:51.600 Campbell Collaboration: Social calculator. It's walks basically put out a model. Get a team of experts and walk around. 410 01:05:51.930 --> 01:06:02.370 Campbell Collaboration: The model look at it and he had it actually is literally Will Graham table looking at looking at the Sam and you comment on it and discuss it and ideas cannot spend an afternoon is doing an LTE walk 411 01:06:04.500 --> 01:06:11.250 Campbell Collaboration: top down approach services for one of the family planning. Once I think there's men and boys next 412 01:06:11.670 --> 01:06:16.950 Campbell Collaboration: Slide this many boys and so they've got a model that color theory, they got it. They got a million things. Here it is. 413 01:06:17.340 --> 01:06:34.020 Campbell Collaboration: And so we're going to go out and use this approach to testing with different causal pathways in our normal level theory service is a much more top down approach, which is, is the is the riptide middle of theory. We're going to go out and test. Test portal chain against interventions. 414 01:06:36.150 --> 01:06:38.520 Campbell Collaboration: top down approach. I think showing 415 01:06:39.870 --> 01:06:43.740 Campbell Collaboration: Here to start with some documents experience analysis and so on. 416 01:06:45.480 --> 01:06:57.720 Campbell Collaboration: Criticizing the protocol. It's mostly done in my career is to sit down and write up to train myself and consulted people. What I would say is don't use signers. So the example. The top is a siloed approach. I do think the 417 01:06:58.110 --> 01:07:03.450 Campbell Collaboration: One that you have in the moment, as opposed to a siloed approach and what I prefer is by using 418 01:07:04.650 --> 01:07:13.050 Campbell Collaboration: A more like in this example here, because if you look at this siloed approach in the top you miss that specific activities links different output. 419 01:07:13.530 --> 01:07:26.790 Campbell Collaboration: Output links to outcomes and so on. We should wear your portal principles are when you've got, what, here's what we're actors, it'll my outputs. You don't know where the, where the areas around where the links are so haven't got calls or principles is stated there so silos. 420 01:07:27.900 --> 01:07:41.760 Campbell Collaboration: Don't seem don't seem to fit your this this this example comes my video I have on there is a change and feel free to go. Look at that. And I have a paper also on a daily basis magic reviews and that shows I think has his example. 421 01:07:43.410 --> 01:07:50.130 Campbell Collaboration: Of iteration was punctuation is you got you got to start somewhere. So then you do start with model or you start with 422 01:07:50.700 --> 01:07:59.670 Campbell Collaboration: The evidence. And so you do is read between the two. I forget which quote this comes from nab is one of the studies. So a logic model bill interests of lay 423 01:07:59.910 --> 01:08:09.060 Campbell Collaboration: Info so it says it's the Washington inspection period emotion dimensions. So it so it says the logic model predictive lead informed by theory found by stakeholders. 424 01:08:09.540 --> 01:08:13.530 Campbell Collaboration: And so on. That's actually Syrian stakeout 425 01:08:14.370 --> 01:08:19.950 Campbell Collaboration: Getting evidence in this ground swell and what we see in practicing most reviews, at least. 426 01:08:20.220 --> 01:08:28.170 Campbell Collaboration: Use this approach they sought out because the Cambodian review, for example, or the pharmaceutical review I showed earlier, they start with theory of change, which 427 01:08:28.620 --> 01:08:41.160 Campbell Collaboration: Is in fact mid level theory and then they come back and be able to present a revised version of the middle theory based on what the evidence has said, so it's interesting approach will actually be there in practice. Anyway, but you could you can adopt. 428 01:08:41.730 --> 01:08:47.130 Campbell Collaboration: Explicitly intentionally iterative approach to the early development at the mid level theory and I do 429 01:09:03.060 --> 01:09:04.650 Susie Burdett: We just lost a few seconds, their 430 01:09:07.950 --> 01:09:09.690 Campbell Collaboration: Own voice, but that's wonderful again. 431 01:09:14.700 --> 01:09:15.540 Campbell Collaboration: Screen again. 432 01:09:23.280 --> 01:09:23.850 Campbell Collaboration: A resume. 433 01:09:25.410 --> 01:09:33.420 Campbell Collaboration: Resume generation. So in reality I think that, you know, we're not being iterative that some more top down some more some more bottom up. 434 01:09:34.620 --> 01:09:36.480 Campbell Collaboration: And found that my daughter. 435 01:09:37.830 --> 01:09:38.850 Campbell Collaboration: My Moto moto 436 01:09:39.990 --> 01:09:41.190 Campbell Collaboration: Theory, we've got 437 01:09:43.680 --> 01:09:44.130 NANCY CARTWRIGHT: See 438 01:09:44.430 --> 01:09:45.090 NANCY CARTWRIGHT: The slides. 439 01:09:46.830 --> 01:09:48.420 Patrick Ward: No, I can't either. 440 01:10:04.050 --> 01:10:05.400 Campbell Collaboration: Nancy, you were saying something. 441 01:10:08.700 --> 01:10:09.120 Susie Burdett: You weren't 442 01:10:10.950 --> 01:10:12.090 Campbell Collaboration: I wasn't okay 443 01:10:12.660 --> 01:10:14.040 NANCY CARTWRIGHT: It was black screen, please. 444 01:10:14.490 --> 01:10:16.200 Campbell Collaboration: Like me again. 445 01:10:35.250 --> 01:10:37.050 Campbell Collaboration: Alright, thank you. Sorry about that. 446 01:10:38.610 --> 01:10:42.120 Campbell Collaboration: I'm in the best room in the house is the best signal in this room so I can do. 447 01:10:43.410 --> 01:10:50.040 Campbell Collaboration: The math modeling is is it is a top down approach did my model. Now I'm going to test it. 448 01:10:51.210 --> 01:10:53.640 Campbell Collaboration: As an example, we have is caught up. 449 01:10:55.080 --> 01:10:59.640 Campbell Collaboration: Very poverty reduction, where they have a model of the impact of 450 01:11:00.810 --> 01:11:05.430 Campbell Collaboration: Event model, the impact of cash of cash transfers on consumption. 451 01:11:06.510 --> 01:11:16.680 Campbell Collaboration: Okay, so that's how those are being developed, how they're being used in the project. So here I am in the studies. So I had identified four categories and again 452 01:11:17.790 --> 01:11:24.450 Campbell Collaboration: You don't necessarily want actually some, some of the studies I've put in more than one of these categories and you may disagree as the capitalization. Sorry. 453 01:11:25.860 --> 01:11:36.810 Campbell Collaboration: So the first uses we're using the studies about my mid level theory. That's what the studies ball and some of them appendix visit. We're going to do this work and the reviews that. What about mid level theory. That's the theory comes at the end of life. 454 01:11:37.950 --> 01:11:46.050 Campbell Collaboration: And may not be ideal, but that's the focus and some of them, some of them now. The other interesting thing to me, but they were a bit of a 455 01:11:46.440 --> 01:11:56.130 Campbell Collaboration: Logical model Nancy can help us out here hopefully is some of them are testing interventions, some testing the theory. So I can either say, Okay, I've got my theories. Right. 456 01:11:56.340 --> 01:12:09.240 Campbell Collaboration: So I'm going to now test interventions to either work or fight against my theory, just right, or some say, Well, I kind of do it, but I don't have it right or not. So now I've got evidence from the interventions. I'm going to actually see my theory is right or not. 457 01:12:10.620 --> 01:12:20.130 Campbell Collaboration: Yeah, she knows where I have used them how to do each of those. I don't actually have a view on work if this makes sense. And this is done so much. I'm very interested in and execute on that. 458 01:12:20.460 --> 01:12:28.200 Campbell Collaboration: And then we have predicting impact when you use it there, Egypt impact in a new setting which is being done by a few of the studies that we have 459 01:12:28.950 --> 01:12:41.280 Campbell Collaboration: Of course you can also use multiple programs selection design. And I think it's a bit of that in particularly impactful program selection. We haven't got any studies that are actually going and designing new programs based on 460 01:12:41.640 --> 01:12:45.810 Campbell Collaboration: On the research findings, which maybe wasn't putting this call anyway. 461 01:12:46.920 --> 01:12:58.410 Campbell Collaboration: So just to go briefly through those different uses. One is testing interventions. So have mid level principles. And so one is if you tell someone to do something. 462 01:12:59.400 --> 01:13:07.560 Campbell Collaboration: They have to have a stock access stuff. I need to do it on assumptions over there. So one of my favorite examples comes to study. I didn't bother him and years ago it was 463 01:13:08.010 --> 01:13:16.290 Campbell Collaboration: A study where a teacher said I'd like one person, why was I don't have any posters in factories, because they don't have any walls so 464 01:13:17.100 --> 01:13:25.080 Campbell Collaboration: He also puzzles Walton posted on the levels so it doesn't have the staff to do stuff were telling him to do the second one is if someone 465 01:13:25.590 --> 01:13:32.040 Campbell Collaboration: tells them to do action x dash API driven decision to do action annex and the example that cosmic star, the idea to convert while no 466 01:13:32.310 --> 01:13:36.900 Campbell Collaboration: Additional counseling in Bangladesh lanta knows establishes us the sample and her book. 467 01:13:37.260 --> 01:13:49.770 Campbell Collaboration: Is that mothers who are being taught him how to pay the fees and raise their children by nutritional counts, there's the fact was that both the husband's of the mother, the mother that I was child that child's father 468 01:13:50.040 --> 01:14:00.480 Campbell Collaboration: And the child's grandmother's neither nor were key decision makers is making closure process about the Child Nutrition and the failure to include them in any advocacy for the for the 469 01:14:01.170 --> 01:14:09.000 Campbell Collaboration: For the nutritional counseling campaign was a critical weakling and causal chain that actually led me to to practice is not changing. 470 01:14:09.300 --> 01:14:17.880 Campbell Collaboration: As resolved, which is no counseling. So people have glad to speak to the people that make the decisions. So as testing interventions testing the liberal principles. 471 01:14:19.200 --> 01:14:31.920 Campbell Collaboration: Of his testing theories says a very simplifies causal chain again back on family planning card information, these to change knowledge to change attitudes as a whole literature evaluation to news to just simplified it here. 472 01:14:32.280 --> 01:14:43.740 Campbell Collaboration: And I would like to imagine it will change your practice. So we've got here the mediators are the steps in the causal chain and the moderators of things effects. 473 01:14:44.160 --> 01:14:52.590 Campbell Collaboration: Of the whole child operates on Nautilus all the strength of the relationship so modern means a lot longer portal train more dresses affect the strength of the arrows. 474 01:14:53.160 --> 01:15:08.670 Campbell Collaboration: And so a single moderator might be appears perfect barrier facilitator who is you could express a single moderation more general terms. So there could be a barrier, the family planning services and products of physical session accessible. 475 01:15:09.060 --> 01:15:17.730 Campbell Collaboration: Or that could be a physicist iota of doorstep little family planning services and products which will lay behind the success in Bangladesh family planning, I can save this. Notice 476 01:15:18.840 --> 01:15:26.970 Campbell Collaboration: But you could you could put on more general terms as availability of family planning services and products, as Nancy said you can be different levels of your mid level principles of 477 01:15:27.540 --> 01:15:33.060 Campbell Collaboration: Assumptions, you can put this in a regression, so that people on the previous slide is actually structural model. 478 01:15:33.390 --> 01:15:44.490 Campbell Collaboration: And so the media is in that model dependent variables and deconstructed equations and the moderator interactive terms in those equations as I have written down here. So you're going to spread that model using the data. 479 01:15:45.450 --> 01:15:53.880 Campbell Collaboration: That you have if you're using curious. Yeah, then identifying combinator which identifies combinations of factors session without come 480 01:15:55.980 --> 01:16:12.600 Campbell Collaboration: Been necessary or sufficient conditions apartment in his condition and I would suggest to run separate to say for each outcome, including measuring outcomes. So I can merge data with moderators, and possibly mediators in the sense of expansion variables, you know, using a QC I approach. 481 01:16:14.370 --> 01:16:16.620 Campbell Collaboration: In terms of predicting impacts the last 482 01:16:18.720 --> 01:16:30.330 Campbell Collaboration: last box as a service, mostly particularly impacts the all of them actually qualitative quantitative, you have a model rest value model you predict the impact of a particular set of 483 01:16:31.680 --> 01:16:43.590 Campbell Collaboration: Pretty particularly says can you run a model like this multi modal franchise model, then you need to quantify the terms of impact in new setting and then use those quantified. 484 01:16:44.970 --> 01:16:47.760 Campbell Collaboration: Access to this might be impacting when you're setting. 485 01:16:48.600 --> 01:16:54.870 Campbell Collaboration: There's only one started explaining states is going to do that and is used is what we should be using modeling of the 486 01:16:55.170 --> 01:17:02.490 Campbell Collaboration: Now, to be out of, out of sample customers. They basically in that is what they're going to do you estimate the model us and they've got 487 01:17:02.820 --> 01:17:09.870 Campbell Collaboration: French observations investment demolishing 18 observations and then test the model against the two who left out. 488 01:17:10.170 --> 01:17:28.410 Campbell Collaboration: Because testing model against interventions you as you used to calibrate model or terroir driven few degrees of freedom testing model against the your observations with word new to calibrate model, although I'm on a qualitative side YouTube specifically identify relevant assumptions. 489 01:17:30.600 --> 01:17:32.370 Campbell Collaboration: Related to the intervention being effective. 490 01:17:33.450 --> 01:17:44.070 Campbell Collaboration: Effectiveness verbal tool for assessing their presence or absence and then applied in the new setting content likelihood of success in your setting. What I'm, what I'm saying here is if you're if 491 01:17:45.870 --> 01:17:53.310 Campbell Collaboration: Studies in the particular impact column and you recognize that it was about positive impact in your setting do that explicitly 492 01:17:53.970 --> 01:18:03.870 Campbell Collaboration: I've counted as I'm reading intervention puzzles. Only one was actually doing that really explicitly. So you need to actually formally identified code or quantify. 493 01:18:04.230 --> 01:18:21.870 Campbell Collaboration: The new sets into which you want to apply the intervention and formally then calculate some prediction of impact in that new setting. And as I am I missing that in the design papers in some of these proposals. Okay. And I am really out of time in terms of the 494 01:18:22.950 --> 01:18:26.580 Campbell Collaboration: In terms of questions. So I have 30 minutes I've used to do 30 minutes 495 01:18:27.600 --> 01:18:33.030 Campbell Collaboration: I, I will just look at the chat and serve as a burning question. I'll just quickly, but let me just 496 01:18:33.390 --> 01:18:46.680 Campbell Collaboration: Run through this very, very briefly, so they're going. Now the best to magic breakout sessions. As I mentioned at the start of something may have missed it. We have a break now to us via our if you're in in 497 01:18:47.310 --> 01:18:55.710 Campbell Collaboration: What is now the only are in two minutes. So basically 15 minutes we return and what will happen is between now between this session finishing 498 01:18:56.340 --> 01:19:04.950 Campbell Collaboration: And then Susie will have moved you to the room where the breakout session is that you're meant to be here. And then there's a break between the two breakout sessions and again 499 01:19:06.870 --> 01:19:09.360 Campbell Collaboration: Same again tomorrow morning, then we will review with 500 01:19:10.470 --> 01:19:19.920 Campbell Collaboration: Each press each breakout session has two or three presentations presented talk about their study for how they're using the liberal theory. Well, they think the value added middle of theory is 501 01:19:20.190 --> 01:19:25.590 Campbell Collaboration: A chair. This doesn't have a chair will then allow an issue classification Q AMP a longer 502 01:19:25.920 --> 01:19:36.330 Campbell Collaboration: Longer session Adrianne able to to come at the end of that session. So we've got questions about what I presented please bring them up in the breakout sessions. I'll be there in 503 01:19:36.720 --> 01:19:43.740 Campbell Collaboration: One have to have a parallel one on one. Want to the others on each time someone else can otherwise discuss it. 504 01:19:44.790 --> 01:19:49.140 Campbell Collaboration: In the, in the penultimate session tomorrow, which is a breakout session small groups. 505 01:19:49.590 --> 01:19:58.290 Campbell Collaboration: We want each person to come down, noting a set of ideas what they've learned what key takeaways are why we know there's a really bad idea, whatever you want to say. 506 01:19:59.130 --> 01:20:15.300 Campbell Collaboration: share your ideas in the group and each group identified three to five lessons which will draw together in the final plenary session tomorrow resolve to our lesson learning activity and this doc share and I mean look at the chat. Anyone comment in the chat box. That'd be quiet as 507 01:20:16.920 --> 01:20:25.680 Campbell Collaboration: Okay, so I asked me to have the presentation just will share the presentations, just wanted my slides have not been checked for copyright. So I just 508 01:20:26.010 --> 01:20:32.670 Campbell Collaboration: Chose the reason I liked so they can't be distributed to Donna copyright check replace them with digital images or was just typing this out. 509 01:20:33.330 --> 01:20:46.470 Campbell Collaboration: But we can share them review you normally don't go further with them. So we're going to to stop now and thank everyone for your patience. Apologies my week connection. And so you can grab a cup of tea carefully. 510 01:20:47.580 --> 01:20:51.510 Campbell Collaboration: Review with 15 minutes in your breakout sessions thank everybody 511 01:21:56.310 --> 01:21:58.740 Susie Burdett: For those still here do feel free to chat if you wish to