You are not seeing this right now.

My userpic decided to hid his crazy hair under Christmas festivity. Enjoy.

I've been talking to hlide recently (yes, in addition to zodttd). hlide has been working on a PS1 emulator for PSP, or at least the CPU emulation. This is a very important thing, IMO, because Sony's firmware 3.0 emulator (which of course only lets you play a handful of games that you must purchase through a PS3 of all things) is causing a number of people to blindly upgrade and lose the ability to play any homebrew (including gpSP). Unfortunately the efforts getting the most attention are by clueless hacks who have been porting PCSX. I was in particular annoyed with the efforts of a certain "Yoshihiro", who has hyped his emulator for months (with the help of a stuffy individual who goes by magixien, more on him later) and demands both praise and respect. He's really pretty clueless when it comes to what's most important here, optimizations.

Fortunately, hlide knows his stuff. I don't know what his experience is with writing dynarecs, but he has some very good ideas. Some are a bit different from mine and incompatible; others aren't. I think that he has given me some very useful practical insights, and I also think that I have helped steer him away from some mistakes as well. So it's a good thing that we've been talking.

Unfortunately, magixien has started hyping hlide instead, and has decided to tag my name with his in the development of this emulator, as can be seen in this thread. He has in fact posted this very thread verbatim no fewer than three times across multiple message boards. It's pretty obvious that he realized Yoshihiro is hopeless (probably in light of the performance of the Sony emulator) and has now shamelessly jumped ships so he can be the front man of someone else. Because of this multiple people have already IMed me asking about my "involvement" in programming hlide's emulator, which is of course nothing.

Anyway, for those who want a technical headache, feel free to read the rest. *ahem* these are some thoughts on optimizing gpSP's dynarec, inspired by hlide. The first was his idea pretty much exactly, the second I'm still working on and is a hybrid of our ideas and experiments.

Dynamically patched memory handlers:

Right now gpSP performs memory read/writes done by the GBA's CPU by using a table to see if it can map these addresses directly to buffers in memory. If it can't, it will perform more specialized actions. hlide's idea is to determine what type of memory access it is, then patch the actual function call to a more specific memory handler to handle this. This handler will have to first determine if the address is still in proper range for it, if not it will repatch. The check to determine if the address is within a static range is actually a little cheaper than doing a full blown generic handler using a table. It will also make those special cases that are not mapped directly to the table much faster. In particular VRAM access will be sped up significantly, meaning that heavy 3D games like Asterix & Obelix should run great after this.

The only problem is that this approach requires a TON of handlers, particularly because on GBA unaligned reads must be handled seperately. This wouldn't be such a big deal if GAS's macro compatabilities weren't extremely minimal, but alas they are (it also wouldn't be a big deal if I could somehow get a C compiler to emit more rigid functions that don't use most registers and do other spilling, etc, but that's not going to happen). So this might be a lot of redundant work and a lot of debugging, but I think it's worth it.

This approach would actually be using self modifying code on gpSP's part, no lie. It's not really ALL bad, since it'd be infrequent. Just have to be sure to use the cache instruction to clear icache for the altered line.


Delayed modified block handling:

Self modifying code is basically a nightmare to emulate. hlide agrees, but he can really afford to ignore it most of the time because it's not so much of a reality on PSP. On GBA it is actually very common, or at least enough to make some games really slow by the current approach of handling it.

For gpSP, detecting self modifying code isn't too expensive: it incurs an additional memory read for every write. This address is based directly off of the address it's writing to, so it's just a few extra instructions. The problem arises in what to do after it's detected: currently, gpSP will flush all of the RAM translation cache, that is, the PSP code that has been generated for the code that was in GBA's RAM (as opposed to its ROM, this is where most code will be). It will then recompile at least enough code to get the thing running again, and probably more shortly after. If a game is constantly writing to an area that it used as code it gets unplayably slow pretty quickly with this method (see: Doom 2). I devised a crude hack to try to minimize it by artificially breaking up blocks so it wouldn't recompile too much, but this was game specific, ugly, and added overhead to the recompiler.

Instead, it'd be great if the modified block didn't actually get recompiled until it was accessed. hlide's idea (which he posted on ps2dev some time ago) was to modify the start of the block so that when jumped to it escaped to a recompiler. I think this is possible for gpSP, but since blocks can overlap a single modified GBA instruction can have multiple blocks which may "enter" it. I think it might be possible to track these using linked list entries, but I'm not sure yet. Hopefully this will work out, because games like this have proven to be a real headache. I think most games behave well enough but some are of this nature.

This is how I compiled VBA:

My roommate told me I can download Visual Studio Professional 2005 for free off of our university. 1.2GB I had it and I tried compiling VBA with the normal project file - I had to manually build the dependencies and install them, or at least that eventually seemed like the natural thing to do for someone with a more Unixy background. I hit a problem with needing unistd.h, but it turned out that that was just me botching the file while trying to get it to compile on mingw, so I grabbed it again from CVS. Then I realized I needed DirectX, so I got the 505MB SDK from Microsoft's website. Turned out that the headers/libraries were 24MB uncompressed, and the examples were some 740MB. Shame on you, Microsoft, I don't need your ridiculously fancy samples.

While searching for information about the unistd "problem" I found a guide on ngemu's forum that revealed to me that there was actually a VS 2005 project file included, so I loaded that up and lo and behold it compiled the dependencies for me, this time. With DirectX in place and the unistd problem taken care of I finally was able to compile VBA. This was of the utmost importance to progressing gpSP (because I'm not good enough to do this all on my own without some hints from other emulators, sometimes)


This is how I fixed Mario & Luigi:

My hours of debugging between gpSP and BoycottAdvance had revealed that Mario & Luigi crashes when an interrupt occurs at a certain point. Two global booleans would alternate between 0 and 1, we'll call the first one A and the second one B. A would normally be 0 most of the time and only 1 for a short period of time. However, when B is 0 (which seems to occur during certain transition scenes) A will be locked at 1 for longer, and the two will thus oscillate. So during those rare periods when A is 1 and an interrupt occurs, I discovered a terrible bug struck.

I first figured that this was a timing problem. Timing problems occur because instructions do not "take" as long in the emulator as they do on a real GBA, this doesn't mean in real time (of course you can't slow down every instruction to 16MHz when you emulate, unless your emulator is just slow) but it means relative to when interrupts occur. Each instruction takes virtual cycles, but to correctly emulate it so it takes the amount it'd actually take on a real GBA is very expensive. So, I give the best approximation I can do with only compile time information. This has in fact helped me get past a part just before the crash in Mario & Luigi, so I figured the crash was a timing bug.

To further back this up, the crash would not happen in Boycott Advance, however when the interrupt struck if I forced "A" to be 1 after it was read into a CPU register (which the debugger allowed me to do) it would crash. So I figured that interrupts are never supposed to happen when A is 1, and that it's a timing flaw exposing a game bug.

However, I soon realized that PSPVBA (PSP port for VBA) has the same bug I did fix with better timing, if you set the "average clock tick" to 1 or 2. PSPVBA basically had even less accurate timing than gpSP does now, which was done to improve performance (as a rather desperate measure). And yet, PSPVBA would not crash at the interrupt crash point, even if you set average clock ticks to all kinds of things. So I was completely confused, all I could reason was that the bug was in both gpSP and Boycott Advance, but it was only showing up in gpSP because of poor timing. A very unlikely explanation, but the best I could reason.

With VBA compiled, I could first confirm that it does in fact not crash with the same timing behavior as PSPVBA (this was the first modification I made). Then, I confirmed that the interrupt does in fact trigger with the "A" bit set, but only with the timing modification! So this in fact seems to confirm my previous explanation. Then, after a good amount of tracking, I found the divergence between VBA and gpSP; it was because of a strange usage of one of the block instructions on ARM.

I can't really be blamed for not having proper emulation of these. I literally reverse engineered games to death to find out how to do this, and I eventually asked gladius about it to confirm what I had done. The documentation just isn't there. Or maybe I should say, none of the ARM documents I've seen, nor GBATEK, correctly describe what these instructions actually do to calculate both their offsets and writeback behavior, they just give a vague conceptually overview (that is far from accurate, anyway). So I broke down and leafed through VBA's source. VBA's CPU emulation is far from clean, even though they use macros for a few things they don't in a lot of places where they easily could; I'd say the source could be done in well under half the line count. So, I manually looked through all the block instruction implementations and wrote down the differences, parameterized on the type of instruction, and modified my macros this way. The end result was actually somewhat cleaner than what gpSP had before, with the macros taking one fewer argument. More importantly, the dynarec isn't emiting more instructions (I didn't count, but I think it might be emiting one or two fewer).

After testing a few things I banged out a couple bugs with the implementation (really only one) in the interpreter, and was able to reimplement them in the PC and PSP dynarecs. Mario & Luigi gets past this point now and I have a tester playing the game.


This is how I fixed save games in Zelda, and hopefully many other games:

This part is actually really easy, now that I have VBA compiled. First, I looked at the games reported buggy in this department, thanks to a very hander GBA ROM lister tool someone gave me (rather forcefully, but I'm glad he did. Must thank him). They all used 8KB EEPROM so I figured the problem was there. I simply had both gpSP and VBA print out (in VBA, to the debug console, which apparently "systemMessage" is doing) the calculated EEPROM addresses in Zelda and realized the upper bits of gpSP's were shifted off correctly. That was simple to fix. It's just a miracle that it took this long to realize. I've had so many problems with EEPROM, I figured I'd have gotten it right by now.

I want to compile VBA so I can hack at it and take the first step in trying to debug certain games, in a long shot attempt to make progress. Doesn't look like that's going to be happening because the authors have made it all but impossible to do on Windows unless you've paid for your compiler. It won't even come close to working on Mingw/msys, Cygwin, or Visual Studio Express.

Right now I just wanted it to have some hope that I could leap into something. But at this point I'm really just grasping at straws.

I don't have friends here. People generally aren't interested in me and I feel rejected on a daily basis, to the point where I've started resenting people more than wishing for their interaction. Any attempt I've made has quickly turned over.

I spent most of my time in my room, cycling various forums. I can't describe how much I hate this. DMZX is boring, lifeless, and ridiculous. The PSP homebrew forums are all worse, filled with selfish imbeciles who are only there to find their own satisfaction and aren't interested in other people. On AIM I get nagged for ROMs and BIOSes several times a day and otherwise people constantly ask me pointless questions because they think I have the answer. All it is is asking for more and more and more from me while not caring even the slightest bit about me. There are a couple small exceptions and I'm clinging to them as best as I can but it's not enough anymore. There's nothing left to talk about and nothing to keep me going.

I've realized that all of this time I've spent online looking for something isn't because there's something to be found, it's because I've never mentally adjusted to what I've left behind. It's everything. There's no one close and important to me that I confide in and rely on, someone whom I'm actually important to. It's more than that, I used to spend a lot of time on IRC but right now the very thought of it sickens me. Really, I have nothing to turn to and I've dried up in terms of working on programming anything now. All I feel is extremely frustrated, tired, bored, bitter, angry, confused, lonely, and lost.

I wanted to just help people, I wanted to stop thinking about myself, but now I can't think at all. I can't find a stable platform to do anything, everything is grasping at things out of reach. There's nothing, I'm completely disconnected.

This is ridiculous, I'm starting to think I was better off being depressed and suicidal all the time, at least then I had a realistic goal in life.

zodttd, the mastermind of psx4all, is leaving the project behind. He'll instead be focusing on working with hlide, which is great news for the PSP scene. It's especially important with Sony's PS1 emulator right around the corner. I'm concerned about this because I know many people will upgrade to it and lose the ability to run our firmware, possibly forever. It's all a race.

But more than that, he also wants to work on gpSP for GP2X. He's made a post about it on the gp32x forums. We talked about it for some time. I'd like to work on it, I just don't have a GP2X and I don't know if I can dedicate myself to it. But I at least brainstormed with him for a while.

We'd corresponded a decent amount in private message, but this was the first time I talked to him online. I liked talking to him, I think we can both learn some decent things from each other. He also represents somewhat of an ideal I have.. he works for a very small company that writes games. Not that I want to write games, but like me he likes keeping things small and modest in work, and likes low level. We both want to do embedded software. It's good to at least find that someone relates with that, and likes doing things just because other people want it done. Everyone else seems to be after making some huge name for themselves and lots of money...

I'm not sure how the gpSP GP2X deal will go though. Then there's the notion of the DC port which is an entirely different matter, and a PS2 port which would be closer to the PSP one.. and a V43 port which would fall under the same category. So much, but I still have to focus on the PSP version.

At least I finally fixed the Mario vs. Donkey Kong bug. I thought it might have been in the contiguous conditional block scanning but convinced myself it wasn't because I tried something to eliminate it. But I did it wrong. So while trying to set things up so I could actually compare the interpreter with the dynarec I stumbled upon a dynarec where it DID work, and it was because of the conditional blocks. So I redid the flags modification checking and it works.

One less thing to worry about.

Don't know how I'll handle Mario & Luigi or Zelda. Those are the big ones. There are other slight problems and speed issues in various games to look into but I'm at least getting closer to my goals for 0.9. I also have to reimplement the unofficial kai features. He actually e-mailed me today, and I was very pleased that he did. He used a machine translation, which I found very amusing since I was telling someone that that was always a viable solution for him. And indeed it was. And he gave me patches, not just source. Excellent. Course, the only thing he added since his last release was something I already fixed because aeolesc told me about it (I wonder if they colaborate too?! :O) but it's the thought that counts.

Anyway. I like talking about all this technical stuff with people in IMs. I hope it lasts a while longer, although I'm afraid our dreams will all fade away. And the prospects of huge projects always scare me. A lot.

The MZX port actually scared me tremendously. I was convinced that what I was doing was impossible, but pushed forward anyway because I was so desperate to find something else to occupy my thoughts.

I think that's how my life will continue. Finding distractions from the chaotic cage of thoughts. I wonder what the next big one will be?

Oh, and I have to learn about Shor's Algorithm more so I can give a presentation on it Tuesday. >_>

I thought about doing something tonight that I was sure I wouldn't ever do, post in add_me. It's because I had this vague sense that there might be someone who is waiting for me to post.. the same way I am waiting for that person. Just one person, no one else.. sometimes I'd prefer no one else existed for me but that person. Who doesn't exist.

Like always, the post would be an appeal. An appeal of sincerity. I just want to try to show people who I am, but I don't think I can ever convey this. I don't think I can really sort out for myself who I am, although it seems pretty clear who I'm not. That's not worth considering.

I decided to look through the posts there to see if that person was making the first move. It was then that I realized I just couldn't say anything. My words don't belong there.. it's a competition. Try to convince other people that you're worth the one they should choose. I know, that's not really fair, it's really about appealing to others in unique ways. But it really isn't. Everyone there seems to fall into a few broad categories, and they all know this.. they're all trying to outdo each other. And for the most part they all have lots of friends. I'm sure Elig understands what I mean by this, and no, it probably isn't fair, but friendship.. people are not an infinitely renewable resource, there's only so much to go around. All the people with lots of friends all want to be friends with each other, because that means getting more and more friends to add to their collection...

Okay, so I don't really want a friend in this sense, because I can't afford to be treated without any consideration. Sure, I ask Es a couple times a week if he wants to do anything and he says he has plans with his real friends, and I keep asking. And waiting for him after class. Same with my roommates and all, although one of them is nice enough to occasionally apologize. Not that I blame her, I've barely known her...

Don't look at me like that because of what I'm saying. I WOULD do as much as I can for other people. It's not because I want people to like me either, it's because I want to be important to others. I've made that the most important thing to me but how can it sustain me when I'm physically incapable of keeping this up? I can't keep doing things that are useful for others without falling out for a while, if I'm not seen by them I'll just stop existing outside of my own chaotic thoughts, what I DO is not enough for all of the time. I can only do what people aren't really ASKING for, no one wants to actually ask me for anything directly. Or let me help them.. they all want to do things themselves. They want to be independent.. It's just.. pride...

I just want to be part of one person's life.

hearts-of-hope (I don't know if you prefer to be called by this handle, I'll go with the given context) if you're reading this, and I doubt it, you've got to know how much I envy you.

You've just got to.

I just slept for about 15 hours. I can't remember the last time I slept for this long. It didn't feel like that long. I had a lot of dreams, and while none of them so much fell into the "nightmare" category they all felt deeply disturbing.

It's really cold in here...

I don't know if I'll ever be able to escape all of the chaotic thoughts that grip my mind every day. I want to, I just don't know how. Sometimes I reach some kind of understanding with them but for the most part it's just hard to deal with. Especially when I'm very tired.. when I'm not on my guard anything can trigger them and I can get stuck inside. It happened today during class and I had to focus really hard to not completely lose it. Other times I'm actually just having conversations inside my mind, conversations I'd like to have with people.. but there aren't any people... not that I don't try to interact with others, there just aren't any conceivable opportunities and usually people don't want to have anything to do with me when I ask.

I still want to try fixing something up in gpSP to achieve better speed in games that write to VRAM a lot. That'd make my beta tester happy. I couldn't get a general solution going along the path I was thinking but I could at least minimize the current cost heavily with some optimization. The problem is that it'd take rewriting some things in ASM. The annoying thing here is that GAS's macro assembler is much weaker than C's preprocessor and for ASM it hurts further, I know you can somehow run the CPP (not C++) on GAS but I haven't figured out how and it just wouldn't really be the same since C is more uniform for writing different types.. so I'll have to have at least 3 different versions hand written out (maybe 6? I don't know yet.. okay probably not)

But it should be done... it's just hard to focus...

I don't even want to play TotA right now. I still haven't beaten it. Over 95 hours logged in, over the course of 9 days.. while still going to class and other obligations, I think. I'm not really sure how I managed that. I want to beat it as soon as I can, it just won't end D: But when I do beat it I don't know how things will be for me.

I'm still locked inside with all of this and there aren't any clear signs of a way out...

I didn't ask for anything much from her, I don't know why she had to ruin my life like this. :/

Where's Elig :(

He used to nag me on AIM to post in my LJ every few days.

I'd nag him to post in his LJ, if he came on AIM.

I miss Elig.

Do you ever wonder what it'd be like if you lost your memory? And you had your livejournal to go back on.

Would make me wish I took better journals, I guess.

Then you'd get to learn about yourself through the same perception that all of your LJ readers have.

Part of me will always desire to lose my memory. If I learn what happened to me as a third party observer it wouldn't pin me down. I'd be able to learn from it so much more easily. And it wouldn't cloud my thoughts each and every day.

It's a little different than wanting to remove something from your memory. It's wanting to reprogram your memories so that you still know them, but they're as if you never directly experienced them.

I wonder if this is one of the ways multiple personalities spawn..