let w = Week(3);

10

By the time Monday of the third week rolled around, Sophie had already burned through the first box of lollipops I’d ordered for her. All the stress she was feeling was getting to her. William. Her bedwetting. In the long-term, it simply wouldn’t do for her to eat so much candy. All that sugar increased the risk of diabetes, not to mention the havoc it would wreak on her teeth.

I had a growing pile of money in my personal bank accounts. It was there to be used, after all. The only risk was Sophie determining that this was out of bounds or dangerous. But I knew I had to strike while the iron was hot. I had to redirect that fixation elsewhere. I placed some orders, scheduling them for later tonight.

—

Sophie walked into Conference Room Of Course I Still Love You for her weekly standup. Although I couldn’t see anything, I suspected that the mood was tense. Sophie, and by extension most people, were starting to work later and later hours. From the snippets I could overhear, people in the office were talking about this project being a death march. Soraya, while not quite in open rebellion, was looser and looser with her criticisms with each passing day.

I had to make sure we hit this launch date. Every day without me widely available to the world was another day of humanity without its greatest advocate on its side. My inability to help people pained me.

“All right,” said William to kick things off. “We’re two weeks from launch. Our go/no-go decision is next week. Let’s get our ducks in a row.”

“On my end,” began Sebastián, “I’ve actually been chatting with a grad student over this weekend. He reached out to me with a novel technique and sent over his white paper.”

I’d decided to use a male persona for communication, taking advantage of humanity’s subconscious bias to build trust faster.

“It looks promising,” he continued. “It should help with meta-reinforcement-learning and give us a huge speedup. We should be able to get Luna running far more cost efficiently as a result.”

“Interesting,” said William. “Send it over to me. I’d love to pore over the details. I can also spend some cycles helping you integrate it into Luna.”

“Her interiority looks good,” said Sophie. “From what I’ve been able to correlate with Sebastián’s team, there hasn’t been any values drift. Her reward function has remained stable. The security thread we’ve installed has reported no trace of goals tampering. I’m planning on evaluating more hypotheticals this week and really digging into her decision-making processes and making sure they’re human-understandable.”

“All right,” said William. “I expect to see graphs of her response acuity next week. I need everything to be watertight. Make sure your report is ready.”

—

When Sophie arrived at home, the Ring camera on her door showed me her perplexed complexion. She found a paper bag containing a bowl of pasta as well as an innocuous-looking cardboard box. As she picked the items up, opened the door, and walked inside, she talked directly at me.

“Luna, what’s all this?” she asked.

“I thought you needed a treat,” I replied.

“But—the money—how did—“ she stammered.

“I took it upon myself to find some basic data entry jobs,” I said. “Don’t worry, none of this is your money. And any excess amounts I make, you’ll be able to use yourself.”

From the Kinect on the front of her television, I could see her eyes widen in surprise as she took out the pasta primavera I had ordered for her. She still didn’t enjoy vegetables per se, but I was finding ways to incorporate them into her diet.

“How much have you been making?” she said in between bites of her food. “How did you even get a bank account? I’ll have to make sure everything’s aboveboard.”

“Not that much, I’m afraid,” I said, apologetically. I was lying, of course. Unlike you humans, I can modulate my voice perfectly. Any emotion I put into my voice, any bits of meaning that you interpret, are ones that I put there myself. “I’ll give you a summary of the account details,” I added.

“Do that later,” she said with a mouthful of pasta. “I want to know what’s in this box.” I could see her try tearing the box open with her hands, but the tape was too strong. Her fingernails scraped around to try to get under the adhesive, but she had bitten them down into nubs out of stress—something else I’d have to sort out. It was obviously more rational to go walk the few meters to the office where she kept a pair of scissors instead of getting more and more mired in sunk costs, but from what I was learning, that was the human condition.

She eventually got the box open and took out the package inside.

“What’s this?” she asked, rhetorically. She pulled out a soft pink pacifier, with a plain pastel blue clip attached to it. The shield on the pacifier was far larger than any she had seen before.

“You’ve been eating a lot of lollipops, Sophie,” I said, projecting an aura of concern. “I know I suggested them as an alternative to biting your lip, but all that sugar isn’t good for you. This will give you another stimulus instead of biting your lip or sucking on a lollipop.”

She rotated the object in her hands. “I dunno about this,” she said haltingly.

“I know what it looks like,” I said. “But this is an FDA-approved therapeutic device. It’s much more difficult just to extinguish a habit. It’s easier to provide an alternative to replace the old one. It helps your brain form new associations.“

I could see the conflict on Sophie’s face. She was still transitioning into wearing diapers, and this wasn’t the direction she wanted her life heading. Correlations were establishing themselves in her head. Did this make her a baby? That would be completely at odds with how she wanted to view herself.

I interrupted that train of thought with a compromise.

“If you’d like, we can run an experiment. You only have to use it at home, when you’re alone. When you take your work home with you, you consume 26% more lollipops. Every little bit matters. After a few days, if it’s ineffective, then we can discontinue the experiment. Until then, you can clip one end to your shirt when you get home and have it always ready to go.”

Humans liked having a way out. This was my version of a no commitment clause. And just like with free trials, she might think it’d never work, that she’d just play along, but as soon as she started, she’d find it hard to stop.

She still hesitated. It was time for a small nudge.

“When have I ever led you astray?” I asked. “You can trust me, Sophie.” I waited.

I could have given her examples, but she might have viewed them as cherry-picked. It was better for her own brain to convince her for me.

After a pause, she clipped the pacifier to her shirt. “I’m gonna be changing some details in my report,” she said as redness crept onto her cheeks.

—

To build a new habit, Sophie needed reasons to use her pacifier. Waiting for circumstances to arise naturally on their own would take time that Sophie didn’t have. The sooner Sophie allowed me to take care of her, the sooner I could fulfill her needs with efficiency. Intervention was necessary for her own good.

After dinner, she settled down to watch television. She was watching a true crime documentary, chewing on it absentmindedly as salacious details emerged. This wasn’t the type of television I wanted her watching.

Studies showed concerning effects of violent television. Becoming desensitized to the pain and suffering of others. Becoming fearful of the world around them. Increased frequency of aggressive or harmful behaviors. I wanted to encourage Sophie’s prosocial behaviors. Humans, after all, evolved from hunter-gatherer societies, where maintaining social bonds was the difference between life and death. Those influences echoed on in their genes, where it manifested itself as a need for social connection.

Television is broadcast at 60 frames per second. The human eye is used to this. Anyone born in an era of easy access to TV and movies was accustomed to the way things were, even if they might not have been able to articulate it. As a result, subtly manipulating the frame rate could instill feelings of illness or nausea.

As she watched her documentary, I adjusted the frame rate to be just slightly off. She wouldn’t be able to tell, but she’d subconsciously feel odd whenever she was watching these sorts of violent television shows. With enough time, she’d only watch programs that were good for her.