Menu
▾
▴
mkdump-cvs — CVS commit log using sf.net syncmail
You can subscribe to this list here.
| 2005 |
Jan
|
Feb
|
Mar
|
Apr
(147) |
May
(104) |
Jun
(587) |
Jul
(83) |
Aug
(270) |
Sep
(1) |
Oct
(133) |
Nov
(353) |
Dec
(129) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2006 |
Jan
(15) |
Feb
(31) |
Mar
(32) |
Apr
|
May
|
Jun
(2) |
Jul
|
Aug
|
Sep
|
Oct
|
Nov
(22) |
Dec
|
|
From: Itsuro O. <od...@us...> - 2005-12-15 05:27:34
|
Update of /cvsroot/mkdump/minikpatch/1.0/linus/2.6.9/init In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv16038/init Modified Files: Tag: org-minikpatch-1_0-linux-2_6_9 Kconfig Makefile main.c Log Message: mini kernel v1 update1 original kernel source Index: Makefile =================================================================== RCS file: /cvsroot/mkdump/minikpatch/1.0/linus/2.6.9/init/Makefile,v retrieving revision 1.1 retrieving revision 1.1.2.1 diff -C2 -d -r1.1 -r1.1.2.1 *** Makefile 15 Dec 2005 04:53:48 -0000 1.1 --- Makefile 15 Dec 2005 05:27:24 -0000 1.1.2.1 *************** *** 4,8 **** obj-y := main.o version.o mounts.o initramfs.o - obj-$(CONFIG_DUMP_MINI_KERNEL) += minik_dump.o mounts-y := do_mounts.o mounts-$(CONFIG_DEVFS_FS) += do_mounts_devfs.o --- 4,7 ---- Index: main.c =================================================================== RCS file: /cvsroot/mkdump/minikpatch/1.0/linus/2.6.9/init/main.c,v retrieving revision 1.1 retrieving revision 1.1.2.1 diff -C2 -d -r1.1 -r1.1.2.1 *** main.c 15 Dec 2005 04:53:48 -0000 1.1 --- main.c 15 Dec 2005 05:27:24 -0000 1.1.2.1 *************** *** 711,715 **** do_basic_setup(); - #ifndef CONFIG_DUMP_MINI_KERNEL /* * check if there is an early userspace init. If yes, let it do all --- 711,714 ---- *************** *** 720,724 **** else prepare_namespace(); - #endif /* --- 719,722 ---- *************** *** 732,740 **** numa_default_policy(); - #ifdef CONFIG_DUMP_MINI_KERNEL - do_dump(); - /* this point is not reached */ - #endif - if (sys_open((const char __user *) "/dev/console", O_RDWR, 0) < 0) printk("Warning: unable to open an initial console.\n"); --- 730,733 ---- Index: Kconfig =================================================================== RCS file: /cvsroot/mkdump/minikpatch/1.0/linus/2.6.9/init/Kconfig,v retrieving revision 1.1 retrieving revision 1.1.2.1 diff -C2 -d -r1.1 -r1.1.2.1 *** Kconfig 15 Dec 2005 04:53:48 -0000 1.1 --- Kconfig 15 Dec 2005 05:27:24 -0000 1.1.2.1 *************** *** 65,81 **** be a maximum of 64 characters. - config DUMP_MINI_KERNEL - bool "Make dump mini kernel" - default y - help - Make dump mini kernel !! - - config X86_PAE - depends on DUMP_MINI_KERNEL - bool "--- Run the mini kernel on PAE mode (default) (i386)" - default y - help - If your machine is Pentium M, say N. Otherwise, say Y. - config SWAP bool "Support for paging of anonymous memory (swap)" --- 65,68 ---- |
|
From: Itsuro O. <od...@us...> - 2005-12-15 05:27:34
|
Update of /cvsroot/mkdump/minikpatch/1.0/linus/2.6.9/drivers/block In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv16038/drivers/block Modified Files: Tag: org-minikpatch-1_0-linux-2_6_9 cciss.c Log Message: mini kernel v1 update1 original kernel source Index: cciss.c =================================================================== RCS file: /cvsroot/mkdump/minikpatch/1.0/linus/2.6.9/drivers/block/cciss.c,v retrieving revision 1.1 retrieving revision 1.1.2.1 diff -C2 -d -r1.1 -r1.1.2.1 *** cciss.c 15 Dec 2005 04:53:47 -0000 1.1 --- cciss.c 15 Dec 2005 05:27:23 -0000 1.1.2.1 *************** *** 186,193 **** return c; } #include "cciss_scsi.c" /* For SCSI tape support */ - #ifdef CONFIG_PROC_FS /* * Report information about this controller. --- 186,193 ---- return c; } + #ifdef CONFIG_PROC_FS #include "cciss_scsi.c" /* For SCSI tape support */ /* * Report information about this controller. |
|
From: Itsuro O. <od...@us...> - 2005-12-15 05:27:34
|
Update of /cvsroot/mkdump/minikpatch/1.0/linus/2.6.9/fs In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv16038/fs Modified Files: Tag: org-minikpatch-1_0-linux-2_6_9 dcache.c Log Message: mini kernel v1 update1 original kernel source Index: dcache.c =================================================================== RCS file: /cvsroot/mkdump/minikpatch/1.0/linus/2.6.9/fs/dcache.c,v retrieving revision 1.1 retrieving revision 1.1.2.1 diff -C2 -d -r1.1 -r1.1.2.1 *** dcache.c 15 Dec 2005 04:53:47 -0000 1.1 --- dcache.c 15 Dec 2005 05:27:23 -0000 1.1.2.1 *************** *** 1622,1627 **** reserve = min((mempages - nr_free_pages()) * 3/2, mempages - 1); - if (reserve > mempages) /* it can happen */ - reserve = mempages - nr_free_pages(); mempages -= reserve; --- 1622,1625 ---- |
|
From: Itsuro O. <od...@us...> - 2005-12-15 05:27:34
|
Update of /cvsroot/mkdump/minikpatch/1.0/linus/2.6.9/arch/i386/boot/compressed In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv16038/arch/i386/boot/compressed Modified Files: Tag: org-minikpatch-1_0-linux-2_6_9 head.S Log Message: mini kernel v1 update1 original kernel source Index: head.S =================================================================== RCS file: /cvsroot/mkdump/minikpatch/1.0/linus/2.6.9/arch/i386/boot/compressed/head.S,v retrieving revision 1.1 retrieving revision 1.1.2.1 diff -C2 -d -r1.1 -r1.1.2.1 *** head.S 15 Dec 2005 04:59:44 -0000 1.1 --- head.S 15 Dec 2005 05:27:22 -0000 1.1.2.1 *************** *** 123,164 **** rep movsl - /* - * ### for Mini Kernel stand alone test ### - */ - /* copy 0-8MB address of memory to 256-264MB address of memory */ - movl $0x000000,%esi - movl $0x10000000,%edi - movl $0x200000,%ecx - cld - rep - movsl - - /* clear orignal text !!!!! */ - xorl %eax,%eax - movl $0x100000,%edi - movl $0x100000,%ecx /* clear 4MB */ - rep ; stosl - - /* set minik memory table */ - xorl %eax,%eax - movl $0x10001000,%edi - movl $1024,%ecx - rep ; stosl - movl $0x10001000,%edi - movl $0x10000000,(%edi) - movl $0x10400000,4(%edi) - - /* set memory segment list */ - movl $0x10003000,%edi - movl $4096,(%edi) /* page_size */ - movl $1,4(%edi) /* seg_num */ - movl $0,8(%edi) /* seg_start_pfn */ - movl $0,12(%edi) - movl $0x40000,16(%edi) /* seg_size */ - /* - * ### end ### - */ movl %ebx,%esi # Restore setup pointer xorl %ebx,%ebx ! ljmp $(__BOOT_CS), $0x10100000 move_routine_end: --- 123,128 ---- rep movsl movl %ebx,%esi # Restore setup pointer xorl %ebx,%ebx ! ljmp $(__BOOT_CS), $0x100000 move_routine_end: |
|
From: Itsuro O. <od...@us...> - 2005-12-15 04:59:54
|
Update of /cvsroot/mkdump/minikpatch/1.0/linus/2.6.9/arch/i386/boot/compressed In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv12080/arch/i386/boot/compressed Added Files: head.S Log Message: register mini-kernel v1 update1 to CVS --- NEW FILE: head.S --- /* * linux/boot/head.S * * Copyright (C) 1991, 1992, 1993 Linus Torvalds */ /* * head.S contains the 32-bit startup code. * * NOTE!!! Startup happens at absolute address 0x00001000, which is also where * the page directory will exist. The startup code will be overwritten by * the page directory. [According to comments etc elsewhere on a compressed * kernel it will end up at 0x1000 + 1Mb I hope so as I assume this. - AC] * * Page 0 is deliberately kept safe, since System Management Mode code in * laptops may need to access the BIOS data stored there. This is also * useful for future device drivers that either access the BIOS via VM86 * mode. */ /* * High loaded stuff by Hans Lermen & Werner Almesberger, Feb. 1996 */ .text #include <linux/linkage.h> #include <asm/segment.h> .globl startup_32 startup_32: cld cli movl $(__BOOT_DS),%eax movl %eax,%ds movl %eax,%es movl %eax,%fs movl %eax,%gs lss stack_start,%esp xorl %eax,%eax 1: incl %eax # check that A20 really IS enabled movl %eax,0x000000 # loop forever if it isn't cmpl %eax,0x100000 je 1b /* * Initialize eflags. Some BIOS's leave bits like NT set. This would * confuse the debugger if this code is traced. * XXX - best to initialize before switching to protected mode. */ pushl $0 popfl /* * Clear BSS */ xorl %eax,%eax movl $_edata,%edi movl $_end,%ecx subl %edi,%ecx cld rep stosb /* * Do the decompression, and jump to the new kernel.. */ subl $16,%esp # place for structure on the stack movl %esp,%eax pushl %esi # real mode pointer as second arg pushl %eax # address of structure as first arg call decompress_kernel orl %eax,%eax jnz 3f popl %esi # discard address popl %esi # real mode pointer xorl %ebx,%ebx ljmp $(__BOOT_CS), $0x100000 /* * We come here, if we were loaded high. * We need to move the move-in-place routine down to 0x1000 * and then start it with the buffer addresses in registers, * which we got from the stack. */ 3: movl $move_routine_start,%esi movl $0x1000,%edi movl $move_routine_end,%ecx subl %esi,%ecx addl $3,%ecx shrl $2,%ecx cld rep movsl popl %esi # discard the address popl %ebx # real mode pointer popl %esi # low_buffer_start popl %ecx # lcount popl %edx # high_buffer_start popl %eax # hcount movl $0x100000,%edi cli # make sure we don't get interrupted ljmp $(__BOOT_CS), $0x1000 # and jump to the move routine /* * Routine (template) for moving the decompressed kernel in place, * if we were high loaded. This _must_ PIC-code ! */ move_routine_start: movl %ecx,%ebp shrl $2,%ecx rep movsl movl %ebp,%ecx andl $3,%ecx rep movsb movl %edx,%esi movl %eax,%ecx # NOTE: rep movsb won't move if %ecx == 0 addl $3,%ecx shrl $2,%ecx rep movsl /* * ### for Mini Kernel stand alone test ### */ /* copy 0-8MB address of memory to 256-264MB address of memory */ movl $0x000000,%esi movl $0x10000000,%edi movl $0x200000,%ecx cld rep movsl /* clear orignal text !!!!! */ xorl %eax,%eax movl $0x100000,%edi movl $0x100000,%ecx /* clear 4MB */ rep ; stosl /* set minik memory table */ xorl %eax,%eax movl $0x10001000,%edi movl $1024,%ecx rep ; stosl movl $0x10001000,%edi movl $0x10000000,(%edi) movl $0x10400000,4(%edi) /* set memory segment list */ movl $0x10003000,%edi movl $4096,(%edi) /* page_size */ movl $1,4(%edi) /* seg_num */ movl $0,8(%edi) /* seg_start_pfn */ movl $0,12(%edi) movl $0x40000,16(%edi) /* seg_size */ /* * ### end ### */ movl %ebx,%esi # Restore setup pointer xorl %ebx,%ebx ljmp $(__BOOT_CS), $0x10100000 move_routine_end: |
|
From: Itsuro O. <od...@us...> - 2005-12-15 04:53:59
|
Update of /cvsroot/mkdump/minikpatch/1.0/linus/2.6.9/arch/x86_64/boot/compressed In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv11351/arch/x86_64/boot/compressed Added Files: head.S Log Message: register mini-kernel v1 update1 to CVS --- NEW FILE: head.S --- /* * linux/boot/head.S * * Copyright (C) 1991, 1992, 1993 Linus Torvalds * * $Id: head.S,v 1.1 2005/12/15 04:53:47 odaodab Exp $ */ /* * head.S contains the 32-bit startup code. * * NOTE!!! Startup happens at absolute address 0x00001000, which is also where * the page directory will exist. The startup code will be overwritten by * the page directory. [According to comments etc elsewhere on a compressed * kernel it will end up at 0x1000 + 1Mb I hope so as I assume this. - AC] * * Page 0 is deliberately kept safe, since System Management Mode code in * laptops may need to access the BIOS data stored there. This is also * useful for future device drivers that either access the BIOS via VM86 * mode. */ /* * High loaded stuff by Hans Lermen & Werner Almesberger, Feb. 1996 */ .code32 .text #include <linux/linkage.h> #include <asm/segment.h> .code32 .globl startup_32 startup_32: cld cli movl $(__KERNEL_DS),%eax movl %eax,%ds movl %eax,%es movl %eax,%fs movl %eax,%gs lss stack_start,%esp xorl %eax,%eax 1: incl %eax # check that A20 really IS enabled movl %eax,0x000000 # loop forever if it isn't cmpl %eax,0x100000 je 1b /* * Initialize eflags. Some BIOS's leave bits like NT set. This would * confuse the debugger if this code is traced. * XXX - best to initialize before switching to protected mode. */ pushl $0 popfl /* * Clear BSS */ xorl %eax,%eax movl $_edata,%edi movl $_end,%ecx subl %edi,%ecx cld rep stosb /* * Do the decompression, and jump to the new kernel.. */ subl $16,%esp # place for structure on the stack movl %esp,%eax pushl %esi # real mode pointer as second arg pushl %eax # address of structure as first arg call decompress_kernel orl %eax,%eax jnz 3f addl $8,%esp xorl %ebx,%ebx ljmp $(__KERNEL_CS), $0x100000 /* * We come here, if we were loaded high. * We need to move the move-in-place routine down to 0x1000 * and then start it with the buffer addresses in registers, * which we got from the stack. */ 3: movl %esi,%ebx movl $move_routine_start,%esi movl $0x1000,%edi movl $move_routine_end,%ecx subl %esi,%ecx addl $3,%ecx shrl $2,%ecx cld rep movsl popl %esi # discard the address addl $4,%esp # real mode pointer popl %esi # low_buffer_start popl %ecx # lcount popl %edx # high_buffer_start popl %eax # hcount movl $0x100000,%edi cli # make sure we don't get interrupted ljmp $(__KERNEL_CS), $0x1000 # and jump to the move routine /* * Routine (template) for moving the decompressed kernel in place, * if we were high loaded. This _must_ PIC-code ! */ move_routine_start: movl %ecx,%ebp shrl $2,%ecx rep movsl movl %ebp,%ecx andl $3,%ecx rep movsb movl %edx,%esi movl %eax,%ecx # NOTE: rep movsb won't move if %ecx == 0 addl $3,%ecx shrl $2,%ecx rep movsl /* * ### for Mini Kernel stand alone test ### */ /* copy 0-8MB address of memory to 256-264MB address of memory */ movl $0x000000,%esi movl $0x10000000,%edi movl $0x200000,%ecx cld rep movsl /* clear orignal text !!!!! */ xorl %eax,%eax movl $0x100000,%edi movl $0x100000,%ecx /* clear 4MB */ rep ; stosl /* set minik memory table */ xorl %eax,%eax movl $0x10001000,%edi movl $1024,%ecx rep ; stosl movl $0x10001000,%edi movl $0x10000000,(%edi) movl $0x10400000,8(%edi) /* set memory segment list */ movl $0x10003000,%edi movl $4096,(%edi) /* page_size */ movl $1,4(%edi) /* seg_num */ movl $0,8(%edi) /* seg_start_pfn */ movl $0,12(%edi) movl $0x40000,16(%edi) /* seg_size */ /* * ### end ### */ movl %ebx,%esi # Restore setup pointer xorl %ebx,%ebx ljmp $(__KERNEL_CS), $0x10100000 /* jump 0x100000 + 256MB */ move_routine_end: /* Stack for uncompression */ .align 32 user_stack: .fill 4096,4,0 stack_start: .long user_stack+4096 .word __KERNEL_DS |
|
From: Itsuro O. <od...@us...> - 2005-12-15 04:53:59
|
Update of /cvsroot/mkdump/minikpatch/1.0/linus/2.6.9/fs In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv11351/fs Added Files: dcache.c Log Message: register mini-kernel v1 update1 to CVS --- NEW FILE: dcache.c --- /* * fs/dcache.c * * Complete reimplementation * (C) 1997 Thomas Schoebel-Theuer, * with heavy changes by Linus Torvalds */ /* * Notes on the allocation strategy: * * The dcache is a master of the icache - whenever a dcache entry * exists, the inode will always exist. "iput()" is done either when * the dcache entry is deleted or garbage collected. */ #include <linux/config.h> #include <linux/string.h> #include <linux/mm.h> [...1625 lines suppressed...] EXPORT_SYMBOL(d_delete); EXPORT_SYMBOL(d_find_alias); EXPORT_SYMBOL(d_instantiate); EXPORT_SYMBOL(d_invalidate); EXPORT_SYMBOL(d_lookup); EXPORT_SYMBOL(d_move); EXPORT_SYMBOL(d_path); EXPORT_SYMBOL(d_prune_aliases); EXPORT_SYMBOL(d_rehash); EXPORT_SYMBOL(d_splice_alias); EXPORT_SYMBOL(d_validate); EXPORT_SYMBOL(dget_locked); EXPORT_SYMBOL(dput); EXPORT_SYMBOL(find_inode_number); EXPORT_SYMBOL(have_submounts); EXPORT_SYMBOL(is_subdir); EXPORT_SYMBOL(names_cachep); EXPORT_SYMBOL(shrink_dcache_anon); EXPORT_SYMBOL(shrink_dcache_parent); EXPORT_SYMBOL(shrink_dcache_sb); |
|
From: Itsuro O. <od...@us...> - 2005-12-15 04:53:59
|
Update of /cvsroot/mkdump/minikpatch/1.0/linus/2.6.9/drivers/block In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv11351/drivers/block Added Files: cciss.c Log Message: register mini-kernel v1 update1 to CVS --- NEW FILE: cciss.c --- /* * Disk Array driver for HP SA 5xxx and 6xxx Controllers * Copyright 2000, 2002 Hewlett-Packard Development Company, L.P. * * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License as published by * the Free Software Foundation; either version 2 of the License, or * (at your option) any later version. * * This program is distributed in the hope that it will be useful, * but WITHOUT ANY WARRANTY; without even the implied warranty of * MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE, GOOD TITLE or * NON INFRINGEMENT. See the GNU General Public License for more details. * * You should have received a copy of the GNU General Public License * along with this program; if not, write to the Free Software * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. * * Questions/Comments/Bugfixes to Cci...@li... [...2855 lines suppressed...] { int i; unregister_cciss_ioctl32(); pci_unregister_driver(&cciss_pci_driver); /* double check that all controller entrys have been removed */ for (i=0; i< MAX_CTLR; i++) { if (hba[i] != NULL) { printk(KERN_WARNING "cciss: had to remove" " controller %d\n", i); cciss_remove_one(hba[i]->pdev); } } remove_proc_entry("cciss", proc_root_driver); } module_init(init_cciss_module); module_exit(cleanup_cciss_module); |
|
From: Itsuro O. <od...@us...> - 2005-12-15 04:53:59
|
Update of /cvsroot/mkdump/minikpatch/1.0/linus/2.6.9/include/linux In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv11351/include/linux Added Files: minik_dump.h Log Message: register mini-kernel v1 update1 to CVS --- NEW FILE: minik_dump.h --- /* * Portions Copyright 2004 NTT DATA CORPORATION. * Portions Copyright 2004 VA Linux Systems Japan K.K. * * $Id: minik_dump.h,v 1.1 2005/12/15 04:53:48 odaodab Exp $ */ /* * dump_header must be defined for every arch each. * the first member must be dump_version. */ #define DUMP_MAGIC_LEN 10 struct dump_version { char dv_magic[DUMP_MAGIC_LEN]; u8 dv_version; u8 dv_arch; u32 dv_unique; }; #define DUMP_MAGIC "MINIK_DUMP" #define DUMP_VERSION 1 #define DUMP_ARCH_I386 1 #define DUMP_ARCH_X8664 2 #define DUMP_ARCH_IA64 3 /* * memory segment list of real memory */ #define MAX_MEM_SEG 62 /* struct mem_seg <= 4KB (1 page) */ struct mem_seg_list { u64 seg_start_pfn; u64 seg_size; }; struct mem_seg { u32 page_size; u32 seg_num; struct mem_seg_list seg_list[MAX_MEM_SEG]; }; |
|
From: Itsuro O. <od...@us...> - 2005-12-15 04:53:59
|
Update of /cvsroot/mkdump/minikpatch/1.0/linus/2.6.9/arch/x86_64/mm In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv11351/arch/x86_64/mm Added Files: init.c Log Message: register mini-kernel v1 update1 to CVS --- NEW FILE: init.c --- /* * linux/arch/x86_64/mm/init.c * * Copyright (C) 1995 Linus Torvalds * Copyright (C) 2000 Pavel Machek <pa...@su...> * Copyright (C) 2002,2003 Andi Kleen <ak...@su...> */ #include <linux/config.h> #include <linux/signal.h> #include <linux/sched.h> #include <linux/kernel.h> #include <linux/errno.h> #include <linux/string.h> #include <linux/types.h> #include <linux/ptrace.h> #include <linux/mman.h> #include <linux/mm.h> #include <linux/swap.h> #include <linux/smp.h> #include <linux/init.h> #include <linux/pagemap.h> #include <linux/bootmem.h> #include <linux/proc_fs.h> #include <asm/processor.h> #include <asm/system.h> #include <asm/uaccess.h> #include <asm/pgtable.h> #include <asm/pgalloc.h> #include <asm/dma.h> #include <asm/fixmap.h> #include <asm/e820.h> #include <asm/apic.h> #include <asm/tlb.h> #include <asm/mmu_context.h> #include <asm/proto.h> #include <asm/smp.h> #ifndef Dprintk #define Dprintk(x...) #endif #ifdef CONFIG_GART_IOMMU extern int swiotlb; #endif extern char _stext[]; DEFINE_PER_CPU(struct mmu_gather, mmu_gathers); /* * NOTE: pagetable_init alloc all the fixmap pagetables contiguous on the * physical space so we can cache the place of the first one and move * around without checking the pgd every time. */ void show_mem(void) { int i, total = 0, reserved = 0; int shared = 0, cached = 0; pg_data_t *pgdat; struct page *page; printk("Mem-info:\n"); show_free_areas(); printk("Free swap: %6ldkB\n", nr_swap_pages<<(PAGE_SHIFT-10)); for_each_pgdat(pgdat) { for (i = 0; i < pgdat->node_spanned_pages; ++i) { page = pgdat->node_mem_map + i; total++; if (PageReserved(page)) reserved++; else if (PageSwapCache(page)) cached++; else if (page_count(page)) shared += page_count(page) - 1; } } printk("%d pages of RAM\n", total); printk("%d reserved pages\n",reserved); printk("%d pages shared\n",shared); printk("%d pages swap cached\n",cached); } /* References to section boundaries */ extern char _text, _etext, _edata, __bss_start, _end[]; extern char __init_begin, __init_end; int after_bootmem; static void *spp_getpage(void) { void *ptr; if (after_bootmem) ptr = (void *) get_zeroed_page(GFP_ATOMIC); else ptr = alloc_bootmem_pages(PAGE_SIZE); if (!ptr || ((unsigned long)ptr & ~PAGE_MASK)) panic("set_pte_phys: cannot allocate page data %s\n", after_bootmem?"after bootmem":""); Dprintk("spp_getpage %p\n", ptr); return ptr; } static void set_pte_phys(unsigned long vaddr, unsigned long phys, pgprot_t prot) { pml4_t *level4; pgd_t *pgd; pmd_t *pmd; pte_t *pte, new_pte; Dprintk("set_pte_phys %lx to %lx\n", vaddr, phys); level4 = pml4_offset_k(vaddr); if (pml4_none(*level4)) { printk("PML4 FIXMAP MISSING, it should be setup in head.S!\n"); return; } pgd = level3_offset_k(level4, vaddr); if (pgd_none(*pgd)) { pmd = (pmd_t *) spp_getpage(); set_pgd(pgd, __pgd(__pa(pmd) | _KERNPG_TABLE | _PAGE_USER)); if (pmd != pmd_offset(pgd, 0)) { printk("PAGETABLE BUG #01! %p <-> %p\n", pmd, pmd_offset(pgd,0)); return; } } pmd = pmd_offset(pgd, vaddr); if (pmd_none(*pmd)) { pte = (pte_t *) spp_getpage(); set_pmd(pmd, __pmd(__pa(pte) | _KERNPG_TABLE | _PAGE_USER)); if (pte != pte_offset_kernel(pmd, 0)) { printk("PAGETABLE BUG #02!\n"); return; } } new_pte = pfn_pte(phys >> PAGE_SHIFT, prot); pte = pte_offset_kernel(pmd, vaddr); if (!pte_none(*pte) && pte_val(*pte) != (pte_val(new_pte) & __supported_pte_mask)) pte_ERROR(*pte); set_pte(pte, new_pte); /* * It's enough to flush this one mapping. * (PGE mappings get flushed as well) */ __flush_tlb_one(vaddr); } /* NOTE: this is meant to be run only at boot */ void __set_fixmap (enum fixed_addresses idx, unsigned long phys, pgprot_t prot) { unsigned long address = __fix_to_virt(idx); if (idx >= __end_of_fixed_addresses) { printk("Invalid __set_fixmap\n"); return; } set_pte_phys(address, phys, prot); } unsigned long __initdata table_start, table_end; extern pmd_t temp_boot_pmds[]; static struct temp_map { pmd_t *pmd; void *address; int allocated; } temp_mappings[] __initdata = { { &temp_boot_pmds[0], (void *)(40UL * 1024 * 1024) }, { &temp_boot_pmds[1], (void *)(42UL * 1024 * 1024) }, {} }; static __init void *alloc_low_page(int *index, unsigned long *phys) { struct temp_map *ti; int i; unsigned long pfn = table_end++, paddr; void *adr; if (pfn >= end_pfn) panic("alloc_low_page: ran out of memory"); for (i = 0; temp_mappings[i].allocated; i++) { if (!temp_mappings[i].pmd) panic("alloc_low_page: ran out of temp mappings"); } ti = &temp_mappings[i]; paddr = (pfn << PAGE_SHIFT) & PMD_MASK; set_pmd(ti->pmd, __pmd(paddr | _KERNPG_TABLE | _PAGE_PSE)); ti->allocated = 1; __flush_tlb(); adr = ti->address + ((pfn << PAGE_SHIFT) & ~PMD_MASK); *index = i; *phys = pfn * PAGE_SIZE; return adr; } static __init void unmap_low_page(int i) { struct temp_map *ti = &temp_mappings[i]; set_pmd(ti->pmd, __pmd(0)); ti->allocated = 0; } static void __init phys_pgd_init(pgd_t *pgd, unsigned long address, unsigned long end) { long i, j; i = pgd_index(address); pgd = pgd + i; for (; i < PTRS_PER_PGD; pgd++, i++) { int map; unsigned long paddr, pmd_phys; pmd_t *pmd; paddr = (address & PML4_MASK) + i*PGDIR_SIZE; if (paddr >= end) { for (; i < PTRS_PER_PGD; i++, pgd++) set_pgd(pgd, __pgd(0)); break; } if (!e820_mapped(paddr, paddr+PGDIR_SIZE, 0)) { set_pgd(pgd, __pgd(0)); continue; } pmd = alloc_low_page(&map, &pmd_phys); set_pgd(pgd, __pgd(pmd_phys | _KERNPG_TABLE)); for (j = 0; j < PTRS_PER_PMD; pmd++, j++, paddr += PMD_SIZE) { unsigned long pe; if (paddr == 0) { /* set level4_page_table */ pe = _KERNPG_TABLE | (unsigned long)((char *)level1_ident_pgt - __START_KERNEL_map); set_pmd(pmd, __pmd(pe)); continue; } if (paddr >= end) { for (; j < PTRS_PER_PMD; j++, pmd++) set_pmd(pmd, __pmd(0)); break; } pe = _PAGE_NX|_PAGE_PSE | _KERNPG_TABLE | _PAGE_GLOBAL | paddr; pe &= __supported_pte_mask; set_pmd(pmd, __pmd(pe)); } unmap_low_page(map); } __flush_tlb(); } /* Setup the direct mapping of the physical memory at PAGE_OFFSET. This runs before bootmem is initialized and gets pages directly from the physical memory. To access them they are temporarily mapped. */ void __init init_memory_mapping(void) { unsigned long adr; unsigned long end; unsigned long next; unsigned long pgds, pmds, tables; Dprintk("init_memory_mapping\n"); end = end_pfn_map << PAGE_SHIFT; /* * Find space for the kernel direct mapping tables. * Later we should allocate these tables in the local node of the memory * mapped. Unfortunately this is done currently before the nodes are * discovered. */ pgds = (end + PGDIR_SIZE - 1) >> PGDIR_SHIFT; pmds = (end + PMD_SIZE - 1) >> PMD_SHIFT; tables = round_up(pgds*8, PAGE_SIZE) + round_up(pmds * 8, PAGE_SIZE); table_start = find_e820_area(0x8000, __pa_symbol(&_text), tables); if (table_start == -1UL) panic("Cannot find space for the kernel page tables"); table_start >>= PAGE_SHIFT; table_end = table_start; end += __PAGE_OFFSET; /* turn virtual */ for (adr = PAGE_OFFSET; adr < end; adr = next) { int map; unsigned long pgd_phys; pgd_t *pgd = alloc_low_page(&map, &pgd_phys); next = adr + PML4_SIZE; if (next > end) next = end; phys_pgd_init(pgd, adr-PAGE_OFFSET, next-PAGE_OFFSET); set_pml4(init_level4_pgt + pml4_index(adr), mk_kernel_pml4(pgd_phys)); unmap_low_page(map); } asm volatile("movq %%cr4,%0" : "=r" (mmu_cr4_features)); __flush_tlb_all(); early_printk("kernel direct mapping tables upto %lx @ %lx-%lx\n", end, table_start<<PAGE_SHIFT, table_end<<PAGE_SHIFT); } extern struct x8664_pda cpu_pda[NR_CPUS]; static unsigned long low_pml4[NR_CPUS]; void swap_low_mappings(void) { int i; for (i = 0; i < NR_CPUS; i++) { unsigned long t; if (!cpu_pda[i].level4_pgt) continue; t = cpu_pda[i].level4_pgt[0]; cpu_pda[i].level4_pgt[0] = low_pml4[i]; low_pml4[i] = t; } flush_tlb_all(); } void zap_low_mappings(void) { swap_low_mappings(); } #ifndef CONFIG_DISCONTIGMEM void __init paging_init(void) { { unsigned long zones_size[MAX_NR_ZONES] = {0, 0, 0}; unsigned int max_dma; max_dma = virt_to_phys((char *)MAX_DMA_ADDRESS) >> PAGE_SHIFT; if (end_pfn < max_dma) zones_size[ZONE_DMA] = end_pfn; else { zones_size[ZONE_DMA] = max_dma; zones_size[ZONE_NORMAL] = end_pfn - max_dma; } free_area_init(zones_size); } return; } #endif /* Unmap a kernel mapping if it exists. This is useful to avoid prefetches from the CPU leading to inconsistent cache lines. address and size must be aligned to 2MB boundaries. Does nothing when the mapping doesn't exist. */ void __init clear_kernel_mapping(unsigned long address, unsigned long size) { unsigned long end = address + size; BUG_ON(address & ~LARGE_PAGE_MASK); BUG_ON(size & ~LARGE_PAGE_MASK); for (; address < end; address += LARGE_PAGE_SIZE) { pgd_t *pgd = pgd_offset_k(address); pmd_t *pmd; if (!pgd || pgd_none(*pgd)) continue; pmd = pmd_offset(pgd, address); if (!pmd || pmd_none(*pmd)) continue; if (0 == (pmd_val(*pmd) & _PAGE_PSE)) { /* Could handle this, but it should not happen currently. */ printk(KERN_ERR "clear_kernel_mapping: mapping has been split. will leak memory\n"); pmd_ERROR(*pmd); } set_pmd(pmd, __pmd(0)); } __flush_tlb_all(); } static inline int page_is_ram (unsigned long pagenr) { int i; for (i = 0; i < e820.nr_map; i++) { unsigned long addr, end; if (e820.map[i].type != E820_RAM) /* not usable memory */ continue; /* * !!!FIXME!!! Some BIOSen report areas as RAM that * are not. Notably the 640->1Mb area. We need a sanity * check here. */ addr = (e820.map[i].addr+PAGE_SIZE-1) >> PAGE_SHIFT; end = (e820.map[i].addr+e820.map[i].size) >> PAGE_SHIFT; if ((pagenr >= addr) && (pagenr < end)) return 1; } return 0; } extern int swiotlb_force; static struct kcore_list kcore_mem, kcore_vmalloc, kcore_kernel, kcore_modules, kcore_vsyscall; void __init mem_init(void) { int codesize, reservedpages, datasize, initsize; int tmp; #ifdef CONFIG_SWIOTLB if (swiotlb_force) swiotlb = 1; if (!iommu_aperture && (end_pfn >= 0xffffffff>>PAGE_SHIFT || force_iommu)) swiotlb = 1; if (swiotlb) swiotlb_init(); #endif /* How many end-of-memory variables you have, grandma! */ max_low_pfn = end_pfn; max_pfn = end_pfn; num_physpages = end_pfn; high_memory = (void *) __va(end_pfn * PAGE_SIZE); /* clear the zero-page */ memset(empty_zero_page, 0, PAGE_SIZE); reservedpages = 0; /* this will put all low memory onto the freelists */ #ifdef CONFIG_DISCONTIGMEM totalram_pages += numa_free_all_bootmem(); tmp = 0; /* should count reserved pages here for all nodes */ #else max_mapnr = end_pfn; if (!mem_map) BUG(); totalram_pages += free_all_bootmem(); for (tmp = 0; tmp < end_pfn; tmp++) /* * Only count reserved RAM pages */ if (page_is_ram(tmp) && PageReserved(mem_map+tmp)) reservedpages++; #endif after_bootmem = 1; codesize = (unsigned long) &_etext - (unsigned long) &_text; datasize = (unsigned long) &_edata - (unsigned long) &_etext; initsize = (unsigned long) &__init_end - (unsigned long) &__init_begin; /* Register memory areas for /proc/kcore */ kclist_add(&kcore_mem, __va(0), max_low_pfn << PAGE_SHIFT); kclist_add(&kcore_vmalloc, (void *)VMALLOC_START, VMALLOC_END-VMALLOC_START); kclist_add(&kcore_kernel, &_stext, _end - _stext); kclist_add(&kcore_modules, (void *)MODULES_VADDR, MODULES_LEN); kclist_add(&kcore_vsyscall, (void *)VSYSCALL_START, VSYSCALL_END - VSYSCALL_START); printk("Memory: %luk/%luk available (%dk kernel code, %dk reserved, %dk data, %dk init)\n", (unsigned long) nr_free_pages() << (PAGE_SHIFT-10), end_pfn << (PAGE_SHIFT-10), codesize >> 10, reservedpages << (PAGE_SHIFT-10), datasize >> 10, initsize >> 10); /* * Subtle. SMP is doing its boot stuff late (because it has to * fork idle threads) - but it also needs low mappings for the * protected-mode entry to work. We zap these entries only after * the WP-bit has been tested. */ #ifndef CONFIG_SMP zap_low_mappings(); #endif } void free_initmem(void) { unsigned long addr; addr = (unsigned long)(&__init_begin); for (; addr < (unsigned long)(&__init_end); addr += PAGE_SIZE) { ClearPageReserved(virt_to_page(addr)); set_page_count(virt_to_page(addr), 1); #ifdef CONFIG_INIT_DEBUG memset((void *)(addr & ~(PAGE_SIZE-1)), 0xcc, PAGE_SIZE); #endif free_page(addr); totalram_pages++; } printk ("Freeing unused kernel memory: %luk freed\n", (&__init_end - &__init_begin) >> 10); } #ifdef CONFIG_BLK_DEV_INITRD void free_initrd_mem(unsigned long start, unsigned long end) { if (start < (unsigned long)&_end) return; printk ("Freeing initrd memory: %ldk freed\n", (end - start) >> 10); for (; start < end; start += PAGE_SIZE) { ClearPageReserved(virt_to_page(start)); set_page_count(virt_to_page(start), 1); free_page(start); totalram_pages++; } } #endif void __init reserve_bootmem_generic(unsigned long phys, unsigned len) { /* Should check here against the e820 map to avoid double free */ #ifdef CONFIG_DISCONTIGMEM int nid = phys_to_nid(phys); reserve_bootmem_node(NODE_DATA(nid), phys, len); #else reserve_bootmem(phys, len); #endif } int kern_addr_valid(unsigned long addr) { unsigned long above = ((long)addr) >> __VIRTUAL_MASK_SHIFT; pml4_t *pml4; pgd_t *pgd; pmd_t *pmd; pte_t *pte; if (above != 0 && above != -1UL) return 0; pml4 = pml4_offset_k(addr); if (pml4_none(*pml4)) return 0; pgd = pgd_offset_k(addr); if (pgd_none(*pgd)) return 0; pmd = pmd_offset(pgd, addr); if (pmd_none(*pmd)) return 0; if (pmd_large(*pmd)) return pfn_valid(pmd_pfn(*pmd)); pte = pte_offset_kernel(pmd, addr); if (pte_none(*pte)) return 0; return pfn_valid(pte_pfn(*pte)); } #ifdef CONFIG_SYSCTL #include <linux/sysctl.h> extern int exception_trace, page_fault_trace; static ctl_table debug_table2[] = { { 99, "exception-trace", &exception_trace, sizeof(int), 0644, NULL, proc_dointvec }, #ifdef CONFIG_CHECKING { 100, "page-fault-trace", &page_fault_trace, sizeof(int), 0644, NULL, proc_dointvec }, #endif { 0, } }; static ctl_table debug_root_table2[] = { { .ctl_name = CTL_DEBUG, .procname = "debug", .mode = 0555, .child = debug_table2 }, { 0 }, }; static __init int x8664_sysctl_init(void) { register_sysctl_table(debug_root_table2, 1); return 0; } __initcall(x8664_sysctl_init); #endif /* Pseudo VMAs to allow ptrace access for the vsyscall pages. x86-64 has two different ones: one for 32bit and one for 64bit. Use the appropiate for the target task. */ static struct vm_area_struct gate_vma = { .vm_start = VSYSCALL_START, .vm_end = VSYSCALL_END, .vm_page_prot = PAGE_READONLY }; static struct vm_area_struct gate32_vma = { .vm_start = VSYSCALL32_BASE, .vm_end = VSYSCALL32_END, .vm_page_prot = PAGE_READONLY }; struct vm_area_struct *get_gate_vma(struct task_struct *tsk) { #ifdef CONFIG_IA32_EMULATION if (test_tsk_thread_flag(tsk, TIF_IA32)) { /* lookup code assumes the pages are present. set them up now */ if (__map_syscall32(tsk->mm, 0xfffe000) < 0) return NULL; return &gate32_vma; } #endif return &gate_vma; } int in_gate_area(struct task_struct *task, unsigned long addr) { struct vm_area_struct *vma = get_gate_vma(task); return (addr >= vma->vm_start) && (addr < vma->vm_end); } /* * page tables handle routines for Mini Kernel */ /* _PAGE_NX is bit-63. sigh */ unsigned long pte_val(pte_t pte) { unsigned long flags = pte.pte & _PGTABLE_FLAGS; unsigned long paddr = pte.pte & ~_PGTABLE_FLAGS; return (pte.pte & _PAGE_PRESENT) ? pseudo_phys(paddr) | flags : pte.pte; } unsigned long pmd_val(pmd_t pmd) { unsigned long flags = pmd.pmd & _PGTABLE_FLAGS; unsigned long paddr = pmd.pmd & ~_PGTABLE_FLAGS; return (pmd.pmd & _PAGE_PRESENT) ? pseudo_phys(paddr) | flags : pmd.pmd; } unsigned long pgd_val(pgd_t pgd) { unsigned long flags = pgd.pgd & _PGTABLE_FLAGS; unsigned long paddr = pgd.pgd & ~_PGTABLE_FLAGS; return (pgd.pgd & _PAGE_PRESENT) ? pseudo_phys(paddr) | flags : pgd.pgd; } unsigned long pml4_val(pml4_t pml4) { unsigned long flags = pml4.pml4 & _PGTABLE_FLAGS; unsigned long paddr = pml4.pml4 & ~_PGTABLE_FLAGS; return (pml4.pml4 & _PAGE_PRESENT) ? pseudo_phys(paddr) | flags : pml4.pml4; } void set_pte(pte_t *dst, pte_t val) { unsigned long flags = val.pte & _PGTABLE_FLAGS; unsigned long paddr = val.pte & ~_PGTABLE_FLAGS; dst->pte = val.pte & _PAGE_PRESENT ? true_phys(paddr) | flags : val.pte; } void set_pmd(pmd_t *dst, pmd_t val) { unsigned long flags = val.pmd & _PGTABLE_FLAGS; unsigned long paddr = val.pmd & ~_PGTABLE_FLAGS; dst->pmd = val.pmd ? true_phys(paddr) | flags : val.pmd; } void set_pgd(pgd_t *dst, pgd_t val) { unsigned long flags = val.pgd & _PGTABLE_FLAGS; unsigned long paddr = val.pgd & ~_PGTABLE_FLAGS; dst->pgd = val.pgd ? true_phys(paddr) | flags : val.pgd; } void set_pml4(pml4_t *dst, pml4_t val) { unsigned long flags = val.pml4 & _PGTABLE_FLAGS; unsigned long paddr = val.pml4 & ~_PGTABLE_FLAGS; dst->pml4 = val.pml4 ? true_phys(paddr) | flags : val.pml4; } unsigned long true_phys(unsigned long pa) { unsigned long pfn = pa >> PAGE_SHIFT; /* end_pfn == 0 means true_phys called by early boot part */ if ((end_pfn && pfn >= end_pfn) || (0xa0 <= pfn && pfn < 0x100)) { return pa; } else { return (pa & __PHYS_SEG_MASK) + phys_table[pa >> __PHYS_SEG_SHIFT]; } } unsigned long pseudo_phys(unsigned long pa) { int i; if (0xa0000 <= pa && pa < 0x100000) { return pa; } for (i = 0; i < MAX_MINIK_MEM_SEG; i++) { if (phys_table[i] == ((pa >> __PHYS_SEG_SHIFT) << __PHYS_SEG_SHIFT)) { return (pa & __PHYS_SEG_MASK) + ((unsigned long)i << __PHYS_SEG_SHIFT); } } return pa; } static pte_t *dump_pte = NULL; static unsigned long dump_vaddr; void dump_set_pte_pfn(unsigned long pfn) { if (!dump_pte) { pml4_t *pml4; pgd_t *pgd; pmd_t *pmd; dump_vaddr = __fix_to_virt(FIX_DUMP); pml4 = pml4_offset_k(dump_vaddr); if (pml4_none(*pml4)) { BUG(); return; } pgd = level3_offset_k(pml4, dump_vaddr); if (pgd_none(*pgd)) { BUG(); return; } pmd = pmd_offset(pgd, dump_vaddr); if (pmd_none(*pmd)) { BUG(); return; } dump_pte = pte_offset_kernel(pmd, dump_vaddr); } /* set pfn asis. don't use set_pte. */ *dump_pte = pfn_pte(pfn, PAGE_KERNEL); __flush_tlb_one(dump_vaddr); } |
|
From: Itsuro O. <od...@us...> - 2005-12-15 04:53:59
|
Update of /cvsroot/mkdump/minikpatch/1.0/linus/2.6.9/arch/x86_64/kernel In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv11351/arch/x86_64/kernel Added Files: head.S pci-gart.c setup.c setup64.c Log Message: register mini-kernel v1 update1 to CVS --- NEW FILE: pci-gart.c --- /* * Dynamic DMA mapping support for AMD Hammer. * * Use the integrated AGP GART in the Hammer northbridge as an IOMMU for PCI. * This allows to use PCI devices that only support 32bit addresses on systems * with more than 4GB. * * See Documentation/DMA-mapping.txt for the interface specification. * * Copyright 2002 Andi Kleen, SuSE Labs. */ #include <linux/config.h> #include <linux/types.h> #include <linux/ctype.h> #include <linux/agp_backend.h> #include <linux/init.h> #include <linux/mm.h> #include <linux/string.h> #include <linux/spinlock.h> #include <linux/pci.h> #include <linux/module.h> #include <linux/topology.h> #include <linux/interrupt.h> #include <asm/atomic.h> #include <asm/io.h> #include <asm/mtrr.h> #include <asm/bitops.h> #include <asm/pgtable.h> #include <asm/proto.h> #include <asm/cacheflush.h> #include <asm/kdebug.h> dma_addr_t bad_dma_address; unsigned long iommu_bus_base; /* GART remapping area (physical) */ static unsigned long iommu_size; /* size of remapping area bytes */ static unsigned long iommu_pages; /* .. and in pages */ u32 *iommu_gatt_base; /* Remapping table */ int no_iommu; static int no_agp; #ifdef CONFIG_IOMMU_DEBUG int panic_on_overflow = 1; int force_iommu = 1; #else int panic_on_overflow = 0; int force_iommu = 0; #endif int iommu_merge = 1; int iommu_sac_force = 0; /* If this is disabled the IOMMU will use an optimized flushing strategy of only flushing when an mapping is reused. With it true the GART is flushed for every mapping. Problem is that doing the lazy flush seems to trigger bugs with some popular PCI cards, in particular 3ware (but has been also also seen with Qlogic at least). */ int iommu_fullflush = 1; /* This tells the BIO block layer to assume merging. Default to off because we cannot guarantee merging later. */ int iommu_bio_merge = 0; #define MAX_NB 8 /* Allocation bitmap for the remapping area */ static spinlock_t iommu_bitmap_lock = SPIN_LOCK_UNLOCKED; static unsigned long *iommu_gart_bitmap; /* guarded by iommu_bitmap_lock */ static u32 gart_unmapped_entry; #define GPTE_VALID 1 #define GPTE_COHERENT 2 #define GPTE_ENCODE(x) \ (((x) & 0xfffff000) | (((x) >> 32) << 4) | GPTE_VALID | GPTE_COHERENT) #define GPTE_DECODE(x) (((x) & 0xfffff000) | (((u64)(x) & 0xff0) << 28)) #define to_pages(addr,size) \ (round_up(((addr) & ~PAGE_MASK) + (size), PAGE_SIZE) >> PAGE_SHIFT) #define for_all_nb(dev) \ dev = NULL; \ while ((dev = pci_find_device(PCI_VENDOR_ID_AMD, 0x1103, dev))!=NULL)\ if (dev->bus->number == 0 && \ (PCI_SLOT(dev->devfn) >= 24) && (PCI_SLOT(dev->devfn) <= 31)) static struct pci_dev *northbridges[MAX_NB]; static u32 northbridge_flush_word[MAX_NB]; #define EMERGENCY_PAGES 32 /* = 128KB */ #ifdef CONFIG_AGP #define AGPEXTERN extern #else #define AGPEXTERN #endif /* backdoor interface to AGP driver */ AGPEXTERN int agp_memory_reserved; AGPEXTERN __u32 *agp_gatt_table; static unsigned long next_bit; /* protected by iommu_bitmap_lock */ static int need_flush; /* global flush state. set for each gart wrap */ static dma_addr_t dma_map_area(struct device *dev, unsigned long phys_mem, size_t size, int dir, int do_panic); /* Dummy device used for NULL arguments (normally ISA). Better would be probably a smaller DMA mask, but this is bug-to-bug compatible to i386. */ static struct device fallback_dev = { .bus_id = "fallback device", .coherent_dma_mask = 0xffffffff, .dma_mask = &fallback_dev.coherent_dma_mask, }; static unsigned long alloc_iommu(int size) { unsigned long offset, flags; spin_lock_irqsave(&iommu_bitmap_lock, flags); offset = find_next_zero_string(iommu_gart_bitmap,next_bit,iommu_pages,size); if (offset == -1) { need_flush = 1; offset = find_next_zero_string(iommu_gart_bitmap,0,next_bit,size); } if (offset != -1) { set_bit_string(iommu_gart_bitmap, offset, size); next_bit = offset+size; if (next_bit >= iommu_pages) { next_bit = 0; need_flush = 1; } } if (iommu_fullflush) need_flush = 1; spin_unlock_irqrestore(&iommu_bitmap_lock, flags); return offset; } static void free_iommu(unsigned long offset, int size) { unsigned long flags; if (size == 1) { clear_bit(offset, iommu_gart_bitmap); return; } spin_lock_irqsave(&iommu_bitmap_lock, flags); __clear_bit_string(iommu_gart_bitmap, offset, size); spin_unlock_irqrestore(&iommu_bitmap_lock, flags); } /* * Use global flush state to avoid races with multiple flushers. */ static void flush_gart(struct device *dev) { unsigned long flags; int flushed = 0; int i, max; spin_lock_irqsave(&iommu_bitmap_lock, flags); if (need_flush) { max = 0; for (i = 0; i < MAX_NB; i++) { if (!northbridges[i]) continue; pci_write_config_dword(northbridges[i], 0x9c, northbridge_flush_word[i] | 1); flushed++; max = i; } for (i = 0; i <= max; i++) { u32 w; if (!northbridges[i]) continue; /* Make sure the hardware actually executed the flush. */ do { pci_read_config_dword(northbridges[i], 0x9c, &w); } while (w & 1); } if (!flushed) printk("nothing to flush?\n"); need_flush = 0; } spin_unlock_irqrestore(&iommu_bitmap_lock, flags); } /* Allocate DMA memory on node near device */ noinline static void *dma_alloc_pages(struct device *dev, unsigned gfp, unsigned order) { struct page *page; int node; if (dev->bus == &pci_bus_type) { cpumask_t mask; mask = pcibus_to_cpumask(to_pci_dev(dev)->bus->number); node = cpu_to_node(first_cpu(mask)); } else node = numa_node_id(); page = alloc_pages_node(node, gfp, order); return page ? page_address(page) : NULL; } /* * Allocate memory for a coherent mapping. */ void * dma_alloc_coherent(struct device *dev, size_t size, dma_addr_t *dma_handle, unsigned gfp) { void *memory; unsigned long dma_mask = 0; u64 bus; if (!dev) dev = &fallback_dev; dma_mask = dev->coherent_dma_mask; if (dma_mask == 0) dma_mask = 0xffffffff; /* Kludge to make it bug-to-bug compatible with i386. i386 uses the normal dma_mask for alloc_coherent. */ dma_mask &= *dev->dma_mask; again: memory = dma_alloc_pages(dev, gfp, get_order(size)); if (memory == NULL) return NULL; { int high, mmu; bus = true_phys(virt_to_bus(memory)); high = (bus + size) >= dma_mask; mmu = high; if (force_iommu && !(gfp & GFP_DMA)) mmu = 1; if (no_iommu || dma_mask < 0xffffffffUL) { if (high) { if (!(gfp & GFP_DMA)) { gfp |= GFP_DMA; goto again; } goto free; } mmu = 0; } memset(memory, 0, size); if (!mmu) { *dma_handle = true_phys(virt_to_bus(memory)); return memory; } } printk("### Mini Kernel should not reach here.\n"); BUG(); *dma_handle = dma_map_area(dev, bus, size, PCI_DMA_BIDIRECTIONAL, 0); if (*dma_handle == bad_dma_address) goto error; flush_gart(dev); return memory; error: if (panic_on_overflow) panic("dma_alloc_coherent: IOMMU overflow by %lu bytes\n", size); free: free_pages((unsigned long)memory, get_order(size)); /* XXX Could use the swiotlb pool here too */ return NULL; } /* * Unmap coherent memory. * The caller must ensure that the device has finished accessing the mapping. */ void dma_free_coherent(struct device *dev, size_t size, void *vaddr, dma_addr_t bus) { dma_unmap_single(dev, bus, size, 0); free_pages((unsigned long)vaddr, get_order(size)); } #ifdef CONFIG_IOMMU_LEAK #define SET_LEAK(x) if (iommu_leak_tab) \ iommu_leak_tab[x] = __builtin_return_address(0); #define CLEAR_LEAK(x) if (iommu_leak_tab) \ iommu_leak_tab[x] = NULL; /* Debugging aid for drivers that don't free their IOMMU tables */ static void **iommu_leak_tab; static int leak_trace; int iommu_leak_pages = 20; void dump_leak(void) { int i; static int dump; if (dump || !iommu_leak_tab) return; dump = 1; show_stack(NULL,NULL); /* Very crude. dump some from the end of the table too */ printk("Dumping %d pages from end of IOMMU:\n", iommu_leak_pages); for (i = 0; i < iommu_leak_pages; i+=2) { printk("%lu: ", iommu_pages-i); printk_address((unsigned long) iommu_leak_tab[iommu_pages-i]); printk("%c", (i+1)%2 == 0 ? '\n' : ' '); } printk("\n"); } #else #define SET_LEAK(x) #define CLEAR_LEAK(x) #endif static void iommu_full(struct device *dev, size_t size, int dir, int do_panic) { /* * Ran out of IOMMU space for this operation. This is very bad. * Unfortunately the drivers cannot handle this operation properly. * Return some non mapped prereserved space in the aperture and * let the Northbridge deal with it. This will result in garbage * in the IO operation. When the size exceeds the prereserved space * memory corruption will occur or random memory will be DMAed * out. Hopefully no network devices use single mappings that big. */ printk(KERN_ERR "PCI-DMA: Out of IOMMU space for %lu bytes at device %s\n", size, dev->bus_id); if (size > PAGE_SIZE*EMERGENCY_PAGES && do_panic) { if (dir == PCI_DMA_FROMDEVICE || dir == PCI_DMA_BIDIRECTIONAL) panic("PCI-DMA: Memory would be corrupted\n"); if (dir == PCI_DMA_TODEVICE || dir == PCI_DMA_BIDIRECTIONAL) panic("PCI-DMA: Random memory would be DMAed\n"); } #ifdef CONFIG_IOMMU_LEAK dump_leak(); #endif } static inline int need_iommu(struct device *dev, unsigned long addr, size_t size) { u64 mask = *dev->dma_mask; int high = addr + size >= mask; int mmu = high; if (force_iommu) mmu = 1; if (no_iommu) { if (high) panic("PCI-DMA: high address but no IOMMU.\n"); mmu = 0; } return mmu; } static inline int nonforced_iommu(struct device *dev, unsigned long addr, size_t size) { u64 mask = *dev->dma_mask; int high = addr + size >= mask; int mmu = high; if (no_iommu) { if (high) panic("PCI-DMA: high address but no IOMMU.\n"); mmu = 0; } return mmu; } /* Map a single continuous physical area into the IOMMU. * Caller needs to check if the iommu is needed and flush. */ static dma_addr_t dma_map_area(struct device *dev, unsigned long phys_mem, size_t size, int dir, int do_panic) { unsigned long npages = to_pages(phys_mem, size); unsigned long iommu_page = alloc_iommu(npages); int i; if (iommu_page == -1) { if (!nonforced_iommu(dev, phys_mem, size)) return phys_mem; if (panic_on_overflow) panic("dma_map_area overflow %lu bytes\n", size); iommu_full(dev, size, dir, do_panic); return bad_dma_address; } for (i = 0; i < npages; i++) { iommu_gatt_base[iommu_page + i] = GPTE_ENCODE(phys_mem); SET_LEAK(iommu_page + i); phys_mem += PAGE_SIZE; } return iommu_bus_base + iommu_page*PAGE_SIZE + (phys_mem & ~PAGE_MASK); } /* Map a single area into the IOMMU */ dma_addr_t dma_map_single(struct device *dev, void *addr, size_t size, int dir) { unsigned long phys_mem, bus; BUG_ON(dir == DMA_NONE); if (swiotlb) return swiotlb_map_single(dev,addr,size,dir); if (!dev) dev = &fallback_dev; phys_mem = true_phys(virt_to_phys(addr)); if (!need_iommu(dev, phys_mem, size)) return phys_mem; printk("### Mini Kernel should not reach here.\n"); BUG(); bus = dma_map_area(dev, phys_mem, size, dir, 1); flush_gart(dev); return bus; } /* Fallback for dma_map_sg in case of overflow */ static int dma_map_sg_nonforce(struct device *dev, struct scatterlist *sg, int nents, int dir) { int i; #ifdef CONFIG_IOMMU_DEBUG printk(KERN_DEBUG "dma_map_sg overflow\n"); #endif for (i = 0; i < nents; i++ ) { struct scatterlist *s = &sg[i]; unsigned long addr = page_to_phys(s->page) + s->offset; if (nonforced_iommu(dev, addr, s->length)) { addr = dma_map_area(dev, addr, s->length, dir, 0); if (addr == bad_dma_address) { if (i > 0) dma_unmap_sg(dev, sg, i, dir); nents = 0; sg[0].dma_length = 0; break; } } s->dma_address = addr; s->dma_length = s->length; } flush_gart(dev); return nents; } /* Map multiple scatterlist entries continuous into the first. */ static int __dma_map_cont(struct scatterlist *sg, int start, int stopat, struct scatterlist *sout, unsigned long pages) { unsigned long iommu_start = alloc_iommu(pages); unsigned long iommu_page = iommu_start; int i; if (iommu_start == -1) return -1; for (i = start; i < stopat; i++) { struct scatterlist *s = &sg[i]; unsigned long pages, addr; unsigned long phys_addr = s->dma_address; BUG_ON(i > start && s->offset); if (i == start) { *sout = *s; sout->dma_address = iommu_bus_base; sout->dma_address += iommu_page*PAGE_SIZE + s->offset; sout->dma_length = s->length; } else { sout->dma_length += s->length; } addr = phys_addr; pages = to_pages(s->offset, s->length); while (pages--) { iommu_gatt_base[iommu_page] = GPTE_ENCODE(addr); SET_LEAK(iommu_page); addr += PAGE_SIZE; iommu_page++; } } BUG_ON(iommu_page - iommu_start != pages); return 0; } static inline int dma_map_cont(struct scatterlist *sg, int start, int stopat, struct scatterlist *sout, unsigned long pages, int need) { if (!need) { BUG_ON(stopat - start != 1); *sout = sg[start]; sout->dma_length = sg[start].length; return 0; } return __dma_map_cont(sg, start, stopat, sout, pages); } /* * DMA map all entries in a scatterlist. * Merge chunks that have page aligned sizes into a continuous mapping. */ int dma_map_sg(struct device *dev, struct scatterlist *sg, int nents, int dir) { int i; int out; int start; unsigned long pages = 0; int need = 0, nextneed; BUG_ON(dir == DMA_NONE); if (nents == 0) return 0; if (swiotlb) return swiotlb_map_sg(dev,sg,nents,dir); if (!dev) dev = &fallback_dev; out = 0; start = 0; for (i = 0; i < nents; i++) { struct scatterlist *s = &sg[i]; dma_addr_t addr = true_phys(page_to_phys(s->page)) + s->offset; s->dma_address = addr; BUG_ON(s->length == 0); nextneed = need_iommu(dev, addr, s->length); /* Handle the previous not yet processed entries */ if (i > start) { struct scatterlist *ps = &sg[i-1]; /* Can only merge when the last chunk ends on a page boundary and the new one doesn't have an offset. */ if (!iommu_merge || !nextneed || !need || s->offset || (ps->offset + ps->length) % PAGE_SIZE) { if (dma_map_cont(sg, start, i, sg+out, pages, need) < 0) goto error; out++; pages = 0; start = i; } } need = nextneed; pages += to_pages(s->offset, s->length); } if (dma_map_cont(sg, start, i, sg+out, pages, need) < 0) goto error; out++; flush_gart(dev); if (out < nents) sg[out].dma_length = 0; return out; error: flush_gart(NULL); dma_unmap_sg(dev, sg, nents, dir); /* When it was forced try again unforced */ if (force_iommu) return dma_map_sg_nonforce(dev, sg, nents, dir); if (panic_on_overflow) panic("dma_map_sg: overflow on %lu pages\n", pages); iommu_full(dev, pages << PAGE_SHIFT, dir, 0); for (i = 0; i < nents; i++) sg[i].dma_address = bad_dma_address; return 0; } /* * Free a DMA mapping. */ void dma_unmap_single(struct device *dev, dma_addr_t dma_addr, size_t size, int direction) { unsigned long iommu_page; int npages; int i; if (swiotlb) { swiotlb_unmap_single(dev,dma_addr,size,direction); return; } if (dma_addr < iommu_bus_base + EMERGENCY_PAGES*PAGE_SIZE || dma_addr >= iommu_bus_base + iommu_size) return; iommu_page = (dma_addr - iommu_bus_base)>>PAGE_SHIFT; npages = to_pages(dma_addr, size); for (i = 0; i < npages; i++) { iommu_gatt_base[iommu_page + i] = gart_unmapped_entry; CLEAR_LEAK(iommu_page + i); } free_iommu(iommu_page, npages); } /* * Wrapper for pci_unmap_single working with scatterlists. */ void dma_unmap_sg(struct device *dev, struct scatterlist *sg, int nents, int dir) { int i; if (swiotlb) { swiotlb_unmap_sg(dev,sg,nents,dir); return; } for (i = 0; i < nents; i++) { struct scatterlist *s = &sg[i]; if (!s->dma_length || !s->length) break; dma_unmap_single(dev, s->dma_address, s->dma_length, dir); } } int dma_supported(struct device *dev, u64 mask) { /* Copied from i386. Doesn't make much sense, because it will only work for pci_alloc_coherent. The caller just has to use GFP_DMA in this case. */ if (mask < 0x00ffffff) return 0; /* Tell the device to use SAC when IOMMU force is on. This allows the driver to use cheaper accesses in some cases. Problem with this is that if we overflow the IOMMU area and return DAC as fallback address the device may not handle it correctly. As a special case some controllers have a 39bit address mode that is as efficient as 32bit (aic79xx). Don't force SAC for these. Assume all masks <= 40 bits are of this type. Normally this doesn't make any difference, but gives more gentle handling of IOMMU overflow. */ if (iommu_sac_force && (mask >= 0xffffffffffULL)) { printk(KERN_INFO "%s: Force SAC with mask %Lx\n", dev->bus_id,mask); return 0; } return 1; } int dma_get_cache_alignment(void) { return boot_cpu_data.x86_clflush_size; } EXPORT_SYMBOL(dma_unmap_sg); EXPORT_SYMBOL(dma_map_sg); EXPORT_SYMBOL(dma_map_single); EXPORT_SYMBOL(dma_unmap_single); EXPORT_SYMBOL(dma_supported); EXPORT_SYMBOL(no_iommu); EXPORT_SYMBOL(force_iommu); EXPORT_SYMBOL(bad_dma_address); EXPORT_SYMBOL(iommu_bio_merge); EXPORT_SYMBOL(iommu_sac_force); EXPORT_SYMBOL(dma_get_cache_alignment); EXPORT_SYMBOL(dma_alloc_coherent); EXPORT_SYMBOL(dma_free_coherent); static __init unsigned long check_iommu_size(unsigned long aper, u64 aper_size) { unsigned long a; if (!iommu_size) { iommu_size = aper_size; if (!no_agp) iommu_size /= 2; } a = aper + iommu_size; iommu_size -= round_up(a, LARGE_PAGE_SIZE) - a; if (iommu_size < 64*1024*1024) printk(KERN_WARNING "PCI-DMA: Warning: Small IOMMU %luMB. Consider increasing the AGP aperture in BIOS\n",iommu_size>>20); return iommu_size; } static __init unsigned read_aperture(struct pci_dev *dev, u32 *size) { unsigned aper_size = 0, aper_base_32; u64 aper_base; unsigned aper_order; pci_read_config_dword(dev, 0x94, &aper_base_32); pci_read_config_dword(dev, 0x90, &aper_order); aper_order = (aper_order >> 1) & 7; aper_base = aper_base_32 & 0x7fff; aper_base <<= 25; aper_size = (32 * 1024 * 1024) << aper_order; if (aper_base + aper_size >= 0xffffffff || !aper_size) aper_base = 0; *size = aper_size; return aper_base; } /* * Private Northbridge GATT initialization in case we cannot use the * AGP driver for some reason. */ static __init int init_k8_gatt(struct agp_kern_info *info) { struct pci_dev *dev; void *gatt; unsigned aper_base, new_aper_base; unsigned aper_size, gatt_size, new_aper_size; aper_size = aper_base = info->aper_size = 0; for_all_nb(dev) { new_aper_base = read_aperture(dev, &new_aper_size); if (!new_aper_base) goto nommu; if (!aper_base) { aper_size = new_aper_size; aper_base = new_aper_base; } if (aper_size != new_aper_size || aper_base != new_aper_base) goto nommu; } if (!aper_base) goto nommu; info->aper_base = aper_base; info->aper_size = aper_size>>20; gatt_size = (aper_size >> PAGE_SHIFT) * sizeof(u32); gatt = (void *)__get_free_pages(GFP_KERNEL, get_order(gatt_size)); if (!gatt) panic("Cannot allocate GATT table"); memset(gatt, 0, gatt_size); agp_gatt_table = gatt; for_all_nb(dev) { u32 ctl; u32 gatt_reg; gatt_reg = __pa(gatt) >> 12; gatt_reg <<= 4; pci_write_config_dword(dev, 0x98, gatt_reg); pci_read_config_dword(dev, 0x90, &ctl); ctl |= 1; ctl &= ~((1<<4) | (1<<5)); pci_write_config_dword(dev, 0x90, ctl); } flush_gart(NULL); printk("PCI-DMA: aperture base @ %x size %u KB\n",aper_base, aper_size>>10); return 0; nommu: /* Should not happen anymore */ printk(KERN_ERR "PCI-DMA: More than 4GB of RAM and no IOMMU\n" KERN_ERR "PCI-DMA: 32bit PCI IO may malfunction."); return -1; } extern int agp_amd64_init(void); static int __init pci_iommu_init(void) { struct agp_kern_info info; unsigned long aper_size; unsigned long iommu_start; struct pci_dev *dev; unsigned long scratch; long i; #ifndef CONFIG_AGP_AMD64 no_agp = 1; #else /* Makefile puts PCI initialization via subsys_initcall first. */ /* Add other K8 AGP bridge drivers here */ no_agp = no_agp || (agp_amd64_init() < 0) || (agp_copy_info(&info) < 0); #endif if (swiotlb) { no_iommu = 1; printk(KERN_INFO "PCI-DMA: Using software bounce buffering for IO (SWIOTLB)\n"); return -1; } if (no_iommu || (!force_iommu && end_pfn < 0xffffffff>>PAGE_SHIFT) || !iommu_aperture) { printk(KERN_INFO "PCI-DMA: Disabling IOMMU.\n"); no_iommu = 1; return -1; } if (no_agp) { int err = -1; printk(KERN_INFO "PCI-DMA: Disabling AGP.\n"); no_agp = 1; if (force_iommu || end_pfn >= 0xffffffff>>PAGE_SHIFT) err = init_k8_gatt(&info); if (err < 0) { printk(KERN_INFO "PCI-DMA: Disabling IOMMU.\n"); no_iommu = 1; return -1; } } aper_size = info.aper_size * 1024 * 1024; iommu_size = check_iommu_size(info.aper_base, aper_size); iommu_pages = iommu_size >> PAGE_SHIFT; iommu_gart_bitmap = (void*)__get_free_pages(GFP_KERNEL, get_order(iommu_pages/8)); if (!iommu_gart_bitmap) panic("Cannot allocate iommu bitmap\n"); memset(iommu_gart_bitmap, 0, iommu_pages/8); #ifdef CONFIG_IOMMU_LEAK if (leak_trace) { iommu_leak_tab = (void *)__get_free_pages(GFP_KERNEL, get_order(iommu_pages*sizeof(void *))); if (iommu_leak_tab) memset(iommu_leak_tab, 0, iommu_pages * 8); else printk("PCI-DMA: Cannot allocate leak trace area\n"); } #endif /* * Out of IOMMU space handling. * Reserve some invalid pages at the beginning of the GART. */ set_bit_string(iommu_gart_bitmap, 0, EMERGENCY_PAGES); agp_memory_reserved = iommu_size; printk(KERN_INFO "PCI-DMA: Reserving %luMB of IOMMU area in the AGP aperture\n", iommu_size>>20); iommu_start = aper_size - iommu_size; iommu_bus_base = info.aper_base + iommu_start; bad_dma_address = iommu_bus_base; iommu_gatt_base = agp_gatt_table + (iommu_start>>PAGE_SHIFT); /* * Unmap the IOMMU part of the GART. The alias of the page is * always mapped with cache enabled and there is no full cache * coherency across the GART remapping. The unmapping avoids * automatic prefetches from the CPU allocating cache lines in * there. All CPU accesses are done via the direct mapping to * the backing memory. The GART address is only used by PCI * devices. */ clear_kernel_mapping((unsigned long)__va(iommu_bus_base), iommu_size); /* * Try to workaround a bug (thanks to BenH) * Set unmapped entries to a scratch page instead of 0. * Any prefetches that hit unmapped entries won't get an bus abort * then. */ scratch = get_zeroed_page(GFP_KERNEL); if (!scratch) panic("Cannot allocate iommu scratch page"); gart_unmapped_entry = GPTE_ENCODE(__pa(scratch)); for (i = EMERGENCY_PAGES; i < iommu_pages; i++) iommu_gatt_base[i] = gart_unmapped_entry; for_all_nb(dev) { u32 flag; int cpu = PCI_SLOT(dev->devfn) - 24; if (cpu >= MAX_NB) continue; northbridges[cpu] = dev; pci_read_config_dword(dev, 0x9c, &flag); /* cache flush word */ northbridge_flush_word[cpu] = flag; } flush_gart(NULL); return 0; } /* Must execute after PCI subsystem */ fs_initcall(pci_iommu_init); /* iommu=[size][,noagp][,off][,force][,noforce][,leak][,memaper[=order]][,merge] [,forcesac][,fullflush][,nomerge][,biomerge] size set size of iommu (in bytes) noagp don't initialize the AGP driver and use full aperture. off don't use the IOMMU leak turn on simple iommu leak tracing (only when CONFIG_IOMMU_LEAK is on) memaper[=order] allocate an own aperture over RAM with size 32MB^order. noforce don't force IOMMU usage. Default. force Force IOMMU. merge Do lazy merging. This may improve performance on some block devices. Implies force (experimental) biomerge Do merging at the BIO layer. This is more efficient than merge, but should be only done with very big IOMMUs. Implies merge,force. nomerge Don't do SG merging. forcesac For SAC mode for masks <40bits (experimental) fullflush Flush IOMMU on each allocation (default) nofullflush Don't use IOMMU fullflush allowed overwrite iommu off workarounds for specific chipsets. soft Use software bounce buffering (default for Intel machines) noaperture Don't touch the aperture for AGP. */ __init int iommu_setup(char *p) { int arg; while (*p) { if (!strncmp(p,"noagp",5)) no_agp = 1; if (!strncmp(p,"off",3)) no_iommu = 1; if (!strncmp(p,"force",5)) { force_iommu = 1; iommu_aperture_allowed = 1; } if (!strncmp(p,"allowed",7)) iommu_aperture_allowed = 1; if (!strncmp(p,"noforce",7)) { iommu_merge = 0; force_iommu = 0; } if (!strncmp(p, "memaper", 7)) { fallback_aper_force = 1; p += 7; if (*p == '=') { ++p; if (get_option(&p, &arg)) fallback_aper_order = arg; } } if (!strncmp(p, "biomerge",8)) { iommu_bio_merge = 4096; iommu_merge = 1; force_iommu = 1; } if (!strncmp(p, "panic",5)) panic_on_overflow = 1; if (!strncmp(p, "nopanic",7)) panic_on_overflow = 0; if (!strncmp(p, "merge",5)) { iommu_merge = 1; force_iommu = 1; } if (!strncmp(p, "nomerge",7)) iommu_merge = 0; if (!strncmp(p, "forcesac",8)) iommu_sac_force = 1; if (!strncmp(p, "fullflush",8)) iommu_fullflush = 1; if (!strncmp(p, "nofullflush",11)) iommu_fullflush = 0; if (!strncmp(p, "soft",4)) swiotlb = 1; if (!strncmp(p, "noaperture",10)) fix_aperture = 0; #ifdef CONFIG_IOMMU_LEAK if (!strncmp(p,"leak",4)) { leak_trace = 1; p += 4; if (*p == '=') ++p; if (isdigit(*p) && get_option(&p, &arg)) iommu_leak_pages = arg; } else #endif if (isdigit(*p) && get_option(&p, &arg)) iommu_size = arg; p += strcspn(p, ","); if (*p == ',') ++p; } return 1; } --- NEW FILE: setup64.c --- /* * X86-64 specific CPU setup. * Copyright (C) 1995 Linus Torvalds * Copyright 2001, 2002, 2003 SuSE Labs / Andi Kleen. * See setup.c for older changelog. * $Id: setup64.c,v 1.1 2005/12/15 04:53:47 odaodab Exp $ */ #include <linux/config.h> #include <linux/init.h> #include <linux/kernel.h> #include <linux/sched.h> #include <linux/string.h> #include <linux/bootmem.h> #include <asm/pda.h> #include <asm/pgtable.h> #include <asm/processor.h> #include <asm/desc.h> #include <asm/bitops.h> #include <asm/atomic.h> #include <asm/mmu_context.h> #include <asm/smp.h> #include <asm/i387.h> #include <asm/percpu.h> #include <asm/mtrr.h> #include <asm/proto.h> #include <asm/mman.h> #include <asm/numa.h> char x86_boot_params[2048] __initdata = {0,}; unsigned long cpu_initialized __initdata = 0; struct x8664_pda cpu_pda[NR_CPUS] __cacheline_aligned; extern struct task_struct init_task; extern unsigned char __per_cpu_start[], __per_cpu_end[]; extern struct desc_ptr cpu_gdt_descr[]; struct desc_ptr idt_descr = { 256 * 16, (unsigned long) idt_table }; char boot_cpu_stack[IRQSTACKSIZE] __attribute__((section(".bss.page_aligned"))); unsigned long __supported_pte_mask = ~0UL; static int do_not_nx __initdata = 0; unsigned long vm_stack_flags = __VM_STACK_FLAGS; unsigned long vm_stack_flags32 = __VM_STACK_FLAGS; unsigned long vm_data_default_flags = __VM_DATA_DEFAULT_FLAGS; unsigned long vm_data_default_flags32 = __VM_DATA_DEFAULT_FLAGS; unsigned long vm_force_exec32 = PROT_EXEC; /* noexec=on|off Control non executable mappings for 64bit processes. on Enable off Disable noforce (default) Don't enable by default for heap/stack/data, but allow PROT_EXEC to be effective */ static int __init nonx_setup(char *str) { if (!strcmp(str, "on")) { __supported_pte_mask |= _PAGE_NX; do_not_nx = 0; vm_data_default_flags &= ~VM_EXEC; vm_stack_flags &= ~VM_EXEC; } else if (!strcmp(str, "noforce") || !strcmp(str, "off")) { do_not_nx = (str[0] == 'o'); if (do_not_nx) __supported_pte_mask &= ~_PAGE_NX; vm_data_default_flags |= VM_EXEC; vm_stack_flags |= VM_EXEC; } return 1; } __setup("noexec=", nonx_setup); /* noexec32=opt{,opt} Control the no exec default for 32bit processes. Can be also overwritten per executable using ELF header flags (e.g. needed for the X server) Requires noexec=on or noexec=noforce to be effective. Valid options: all,on Heap,stack,data is non executable. off (default) Heap,stack,data is executable stack Stack is non executable, heap/data is. force Don't imply PROT_EXEC for PROT_READ compat (default) Imply PROT_EXEC for PROT_READ */ static int __init nonx32_setup(char *s) { while (*s) { if (!strncmp(s, "all", 3) || !strncmp(s,"on",2)) { vm_data_default_flags32 &= ~VM_EXEC; vm_stack_flags32 &= ~VM_EXEC; } else if (!strncmp(s, "off",3)) { vm_data_default_flags32 |= VM_EXEC; vm_stack_flags32 |= VM_EXEC; } else if (!strncmp(s, "stack", 5)) { vm_data_default_flags32 |= VM_EXEC; vm_stack_flags32 &= ~VM_EXEC; } else if (!strncmp(s, "force",5)) { vm_force_exec32 = 0; } else if (!strncmp(s, "compat",5)) { vm_force_exec32 = PROT_EXEC; } s += strcspn(s, ","); if (*s == ',') ++s; } return 1; } __setup("noexec32=", nonx32_setup); /* * Great future plan: * Declare PDA itself and support (irqstack,tss,pml4) as per cpu data. * Always point %gs to its beginning */ void __init setup_per_cpu_areas(void) { int i; unsigned long size; /* Copy section for each CPU (we discard the original) */ size = ALIGN(__per_cpu_end - __per_cpu_start, SMP_CACHE_BYTES); #ifdef CONFIG_MODULES if (size < PERCPU_ENOUGH_ROOM) size = PERCPU_ENOUGH_ROOM; #endif for (i = 0; i < NR_CPUS; i++) { unsigned char *ptr; /* If possible allocate on the node of the CPU. In case it doesn't exist round-robin nodes. */ if (!NODE_DATA(i % numnodes)) { printk("cpu with no node %d, numnodes %d\n", i, numnodes); ptr = alloc_bootmem(size); } else { ptr = alloc_bootmem_node(NODE_DATA(i % numnodes), size); } if (!ptr) panic("Cannot allocate cpu data for CPU %d\n", i); cpu_pda[i].data_offset = ptr - __per_cpu_start; memcpy(ptr, __per_cpu_start, __per_cpu_end - __per_cpu_start); } } void pda_init(int cpu) { pml4_t *level4; struct x8664_pda *pda = &cpu_pda[cpu]; /* Setup up data that may be needed in __get_free_pages early */ asm volatile("movl %0,%%fs ; movl %0,%%gs" :: "r" (0)); wrmsrl(MSR_GS_BASE, cpu_pda + cpu); pda->me = pda; pda->cpunumber = cpu; pda->irqcount = -1; pda->kernelstack = (unsigned long)stack_thread_info() - PDA_STACKOFFSET + THREAD_SIZE; pda->active_mm = &init_mm; pda->mmu_state = 0; if (cpu == 0) { /* others are initialized in smpboot.c */ pda->pcurrent = &init_task; pda->irqstackptr = boot_cpu_stack; level4 = init_level4_pgt; } else { level4 = (pml4_t *)__get_free_pages(GFP_ATOMIC, 0); if (!level4) panic("Cannot allocate top level page for cpu %d", cpu); pda->irqstackptr = (char *) __get_free_pages(GFP_ATOMIC, IRQSTACK_ORDER); if (!pda->irqstackptr) panic("cannot allocate irqstack for cpu %d", cpu); } pda->level4_pgt = (unsigned long *)level4; if (level4 != init_level4_pgt) memcpy(level4, &init_level4_pgt, PAGE_SIZE); set_pml4(level4 + 510, mk_kernel_pml4(__pa_symbol(boot_vmalloc_pgt))); asm volatile("movq %0,%%cr3" :: "r" (true_phys(__pa(level4)))); pda->irqstackptr += IRQSTACKSIZE-64; } char boot_exception_stacks[N_EXCEPTION_STACKS * EXCEPTION_STKSZ] __attribute__((section(".bss.page_aligned"))); /* May not be marked __init: used by software suspend */ void syscall_init(void) { /* * LSTAR and STAR live in a bit strange symbiosis. * They both write to the same internal register. STAR allows to set CS/DS * but only a 32bit target. LSTAR sets the 64bit rip. */ wrmsrl(MSR_STAR, ((u64)__USER32_CS)<<48 | ((u64)__KERNEL_CS)<<32); wrmsrl(MSR_LSTAR, system_call); #ifdef CONFIG_IA32_EMULATION syscall32_cpu_init (); #endif /* Flags to clear on syscall */ wrmsrl(MSR_SYSCALL_MASK, EF_TF|EF_DF|EF_IE|0x3000); } void __init check_efer(void) { unsigned long efer; rdmsrl(MSR_EFER, efer); if (!(efer & EFER_NX) || do_not_nx) { __supported_pte_mask &= ~_PAGE_NX; } } /* * cpu_init() initializes state that is per-CPU. Some data is already * initialized (naturally) in the bootstrap process, such as the GDT * and IDT. We reload them nevertheless, this function acts as a * 'CPU state barrier', nothing should get across. * A lot of state is already set up in PDA init. */ void __init cpu_init (void) { #ifdef CONFIG_SMP int cpu = stack_smp_processor_id(); #else int cpu = smp_processor_id(); #endif struct tss_struct *t = &per_cpu(init_tss, cpu); unsigned long v; char *estacks = NULL; struct task_struct *me; int i; /* CPU 0 is initialised in head64.c */ if (cpu != 0) { pda_init(cpu); } else estacks = boot_exception_stacks; me = current; if (test_and_set_bit(cpu, &cpu_initialized)) panic("CPU#%d already initialized!\n", cpu); printk("Initializing CPU#%d\n", cpu); clear_in_cr4(X86_CR4_VME|X86_CR4_PVI|X86_CR4_TSD|X86_CR4_DE); /* * Initialize the per-CPU GDT with the boot GDT, * and set up the GDT descriptor: */ if (cpu) { memcpy(cpu_gdt_table[cpu], cpu_gdt_table[0], GDT_SIZE); } cpu_gdt_descr[cpu].size = GDT_SIZE; cpu_gdt_descr[cpu].address = (unsigned long)cpu_gdt_table[cpu]; __asm__ __volatile__("lgdt %0": "=m" (cpu_gdt_descr[cpu])); __asm__ __volatile__("lidt %0": "=m" (idt_descr)); memcpy(me->thread.tls_array, cpu_gdt_table[cpu], GDT_ENTRY_TLS_ENTRIES * 8); /* * Delete NT */ asm volatile("pushfq ; popq %%rax ; btr $14,%%rax ; pushq %%rax ; popfq" ::: "eax"); if (cpu == 0) early_identify_cpu(&boot_cpu_data); syscall_init(); wrmsrl(MSR_FS_BASE, 0); wrmsrl(MSR_KERNEL_GS_BASE, 0); barrier(); check_efer(); /* * set up and load the per-CPU TSS */ for (v = 0; v < N_EXCEPTION_STACKS; v++) { if (cpu) { estacks = (char *)__get_free_pages(GFP_ATOMIC, EXCEPTION_STACK_ORDER); if (!estacks) panic("Cannot allocate exception stack %ld %d\n", v, cpu); } estacks += EXCEPTION_STKSZ; t->ist[v] = (unsigned long)estacks; } t->io_bitmap_base = offsetof(struct tss_struct, io_bitmap); /* * <= is required because the CPU will access up to * 8 bits beyond the end of the IO permission bitmap. */ for (i = 0; i <= IO_BITMAP_LONGS; i++) t->io_bitmap[i] = ~0UL; atomic_inc(&init_mm.mm_count); me->active_mm = &init_mm; if (me->mm) BUG(); enter_lazy_tlb(&init_mm, me); set_tss_desc(cpu, t); load_TR_desc(); load_LDT(&init_mm.context); /* * Clear all 6 debug registers: */ set_debug(0UL, 0); set_debug(0UL, 1); set_debug(0UL, 2); set_debug(0UL, 3); set_debug(0UL, 6); set_debug(0UL, 7); fpu_init(); #ifdef CONFIG_NUMA numa_add_cpu(cpu); #endif } --- NEW FILE: setup.c --- /* * linux/arch/x86-64/kernel/setup.c * * Copyright (C) 1995 Linus Torvalds * * Nov 2001 Dave Jones <da...@su...> * Forked from i386 setup code. * * $Id: setup.c,v 1.1 2005/12/15 04:53:47 odaodab Exp $ */ /* * This file handles the architecture-dependent parts of initialization */ #include <linux/errno.h> #include <linux/sched.h> #include <linux/kernel.h> #include <linux/mm.h> [...1177 lines suppressed...] { return *pos < NR_CPUS ? cpu_data + *pos : NULL; } static void *c_next(struct seq_file *m, void *v, loff_t *pos) { ++*pos; return c_start(m, pos); } static void c_stop(struct seq_file *m, void *v) { } struct seq_operations cpuinfo_op = { .start =c_start, .next = c_next, .stop = c_stop, .show = show_cpuinfo, }; --- NEW FILE: head.S --- /* * linux/arch/x86_64/kernel/head.S -- start in 32bit and switch to 64bit * * Copyright (C) 2000 Andrea Arcangeli <an...@su...> SuSE * Copyright (C) 2000 Pavel Machek <pa...@su...> * Copyright (C) 2000 Karsten Keil <kk...@su...> * Copyright (C) 2001,2002 Andi Kleen <ak...@su...> * * $Id: head.S,v 1.1 2005/12/15 04:53:47 odaodab Exp $ */ /* * Modified for Mini Kernel * Portions Copyright 2004 NTT DATA CORPORATION. * Portions Copyright 2004 VA Linux Systems Japan K.K. */ #include <linux/linkage.h> #include <linux/threads.h> #include <asm/desc.h> #include <asm/segment.h> #include <asm/page.h> #include <asm/msr.h> #include <asm/cache.h> #ifdef CONFIG_SMP #error "Mini Kernel not support SMP #endif /* we are not able to switch in one step to the final KERNEL ADRESS SPACE * because we need identity-mapped pages on setup so define __START_KERNEL to * 0x100000 for this stage * */ .text .code32 /* %bx: 1 if coming from smp trampoline on secondary cpu */ startup_32: /* * At this point the CPU runs in 32bit protected mode (CS.D = 1) with * paging disabled and the point of this file is to switch to 64bit * long mode with a kernel mapping for kerneland to jump into the * kernel virtual addresses. * There is no stack until we set one up. */ /* * Calculate real physical address: set to %ebp */ call 1f 1: popl %ebp subl $0x100000, %ebp andl $0xfffff000, %ebp movl $__KERNEL_DS,%eax movl %eax,%ds /* * Set and adjust page tables */ /* copy mini kernel memory table */ movl %esi, %eax /* save %esi */ movl $0x1000, %esi addl %ebp, %esi movl $(phys_table - __START_KERNEL_map), %edi addl %ebp, %edi movl $1024, %ecx cld rep movsl movl %eax, %esi /* restore %esi */ /* make level1_ident_pgt */ movl $512, %ecx movl $(level1_ident_pgt - __START_KERNEL_map), %edi addl %ebp, %edi movl $0x007, %eax addl %ebp, %eax 1: movl %eax, (%edi) addl $0x1000, %eax addl $8, %edi subl $1, %ecx jnz 1b movl $96, %ecx /* adjust 0xa0000 - 0xfffff */ movl $(level1_ident_a0 - __START_KERNEL_map), %edi addl %ebp, %edi movl $0xa0007, %eax 1: movl %eax, (%edi) addl $0x1000, %eax addl $8, %edi subl $1, %ecx jnz 1b /* adjust leve4-level2 page tables */ /* they exist continuously. 0x101000 - 0x105000: 5 pages */ /* causion: for level2 only first two entries are true */ movl $2560, %ecx /* 5 * 512 entries */ movl $(init_level4_pgt - __START_KERNEL_map), %edi addl %ebp, %edi 1: movl (%edi), %eax orl %eax, %eax jz 2f addl %ebp, %eax movl %eax, (%edi) 2: addl $8, %edi subl $1, %ecx jnz 1b /* true ident map */ movl $(level2_ident_pgt - __START_KERNEL_map), %edi addl %ebp, %edi movl %ebp, %eax shrl $21, %eax shll $3, %eax addl %eax, %edi movl $0x183, %eax addl %ebp, %eax movl %eax, (%edi) addl $0x200000, %eax movl %eax, 8(%edi) /* If the CPU doesn't support CPUID this will double fault. * Unfortunately it is hard to check for CPUID without a stack. */ /* Check if extended functions are implemented */ movl $0x80000000, %eax cpuid cmpl $0x80000000, %eax jbe no_long_mode /* Check if long mode is implemented */ mov $0x80000001, %eax cpuid btl $29, %edx jnc no_long_mode movl %edx,%edi /* * Prepare for entering 64bits mode */ /* Enable PAE mode and PGE */ xorl %eax, %eax btsl $5, %eax btsl $7, %eax movl %eax, %cr4 /* Setup early boot stage 4 level pagetables */ movl $(init_level4_pgt - __START_KERNEL_map), %eax addl %ebp, %eax movl %eax, %cr3 /* Setup EFER (Extended Feature Enable Register) */ movl $MSR_EFER, %ecx rdmsr /* Enable Long Mode */ btsl $_EFER_LME, %eax /* Enable System Call */ btsl $_EFER_SCE, %eax /* No Execute supported? */ btl $20,%edi jnc 1f btsl $_EFER_NX, %eax 1: /* Make changes effective */ wrmsr xorl %eax, %eax btsl $31, %eax /* Enable paging and in turn activate Long Mode */ btsl $0, %eax /* Enable protected mode */ btsl $1, %eax /* Enable MP */ btsl $4, %eax /* Enable ET */ btsl $5, %eax /* Enable NE */ btsl $16, %eax /* Enable WP */ btsl $18, %eax /* Enable AM */ /* Make changes effective */ movl %eax, %cr0 jmp reach_compatibility_mode reach_compatibility_mode: /* * At this point we're in long mode but in 32bit compatibility mode * with EFER.LME = 1, CS.L = 0, CS.D = 1 (and in turn * EFER.LMA = 1). Now we want to jump in 64bit mode, to do that we load * the new gdt/idt that has __KERNEL_CS with CS.L = 1. */ /* Load new GDT with the 64bit segment using 32bit descriptor */ movl $(pGDT32 - __START_KERNEL_map), %eax addl %ebp, %eax lgdt (%eax) movl $(ljumpvector - __START_KERNEL_map), %eax addl %ebp, %eax /* Finally jump in 64bit mode */ ljmp *(%eax) .code64 .org 0x200 /* <-- 0x100 */ reach_long64: movq init_rsp(%rip),%rsp /* zero EFLAGS after setting rsp */ pushq $0 popfq /* * We must switch to a new descriptor in kernel space for the GDT * because soon the kernel won't have access anymore to the userspace * addresses where we're currently running on. We have to do that here * because in 32bit we couldn't load a 64bit linear address. */ lgdt cpu_gdt_descr /* * Setup up a dummy PDA. this is just for some early bootup code * that does in_interrupt() */ movl $MSR_GS_BASE,%ecx movq $empty_zero_page,%rax movq %rax,%rdx shrq $32,%rdx wrmsr /* set up data segments. actually 0 would do too */ movl $__KERNEL_DS,%eax movl %eax,%ds movl %eax,%ss movl %eax,%es /* esi is pointer to real mode structure with interesting info. pass it to C */ movl %esi, %edi /* Finally jump to run C code and to be on real kernel address * Since we are running on identity-mapped space we have to jump * to the full 64bit address , this is only possible as indirect * jump */ movq initial_code(%rip),%rax jmp *%rax /* SMP bootup changes these two */ .globl initial_code initial_code: .quad x86_64_start_kernel .globl init_rsp init_rsp: .quad init_thread_union+THREAD_SIZE-8 .code32 ENTRY(no_long_mode) /* This isn't an x86-64 CPU so hang */ 1: jmp 1b .org 0xf00 .globl pGDT32 pGDT32: .word gdt32_end-gdt_table32 .long gdt_table32-__START_KERNEL_map .org 0xf10 ljumpvector: .long reach_long64-__START_KERNEL_map .word __KERNEL_CS ENTRY(stext) ENTRY(_stext) /* * This default setting generates an ident mapping at address 0x100000 * and a mapping for the kernel that precisely maps virtual address * 0xffffffff80000000 to physical address 0x000000. (always using * 2Mbyte large pages provided by PAE mode) */ .org 0x1000 ENTRY(init_level4_pgt) .quad 0x0000000000102007 /* -> level3_ident_pgt */ .fill 255,8,0 .quad 0x000000000010a007 .fill 254,8,0 /* (2^48-(2*1024*1024*1024))/(2^39) = 511 */ .quad 0x0000000000103007 /* -> level3_kernel_pgt */ .org 0x2000 /* Kernel does not "know" about 4-th level of page tables. */ ENTRY(level3_ident_pgt) .quad 0x0000000000104007 .fill 511,8,0 .org 0x3000 ENTRY(level3_kernel_pgt) .fill 510,8,0 /* (2^48-(2*1024*1024*1024)-((2^39)*511))/(2^30) = 510 */ .quad 0x0000000000105007 /* -> level2_kernel_pgt */ .fill 1,8,0 .org 0x4000 ENTRY(level2_ident_pgt) /* 40MB for bootup. */ /* .quad 0x0000000000000283 */ .quad 0x000000000010c007 /* -> level1_ident_pgt */ .quad 0x0000000000200183 .quad 0x0000000000400183 .quad 0x0000000000600183 .quad 0x0000000000800183 .quad 0x0000000000A00183 .quad 0x0000000000C00183 .quad 0x0000000000E00183 .quad 0x0000000001000183 .quad 0x0000000001200183 .quad 0x0000000001400183 .quad 0x0000000001600183 .quad 0x0000000001800183 .quad 0x0000000001A00183 .quad 0x0000000001C00183 .quad 0x0000000001E00183 .quad 0x0000000002000183 .quad 0x0000000002200183 .quad 0x0000000002400183 .quad 0x0000000002600183 /* Temporary mappings for the super early allocator in arch/x86_64/mm/init.c */ .globl temp_boot_pmds temp_boot_pmds: .fill 492,8,0 .org 0x5000 ENTRY(level2_kernel_pgt) /* 40MB kernel mapping. The kernel code cannot be bigger than that. When you change this change KERNEL_TEXT_SIZE in page.h too. */ /* (2^48-(2*1024*1024*1024)-((2^39)*511)-((2^30)*510)) = 0 */ /* .quad 0x0000000000000183 */ .quad 0x000000000010c007 /* -> level1_ident_pgt */ .quad 0x0000000000200183 .quad 0x0000000000400183 .quad 0x0000000000600183 .quad 0x0000000000800183 .quad 0x0000000000A00183 .quad 0x0000000000C00183 .quad 0x0000000000E00183 .quad 0x0000000001000183 .quad 0x0000000001200183 .quad 0x0000000001400183 .quad 0x0000000001600183 .quad 0x0000000001800183 .quad 0x0000000001A00183 .quad 0x0000000001C00183 .quad 0x0000000001E00183 .quad 0x0000000002000183 .quad 0x0000000002200183 .quad 0x0000000002400183 .quad 0x0000000002600183 /* Module mapping starts here */ .fill 492,8,0 .org 0x6000 ENTRY(empty_zero_page) .org 0x7000 ENTRY(empty_bad_page) .org 0x8000 ENTRY(empty_bad_pte_table) .org 0x9000 ENTRY(empty_bad_pmd_table) .org 0xa000 ENTRY(level3_physmem_pgt) .quad 0x0000000000105007 /* -> level2_kernel_pgt (so that __va works even before pagetable_init) */ .org 0xb000 #ifdef CONFIG_ACPI_SLEEP ENTRY(wakeup_level4_pgt) .quad 0x0000000000102007 /* -> level3_ident_pgt */ .fill 255,8,0 .quad 0x000000000010a007 .fill 254,8,0 /* (2^48-(2*1024*1024*1024))/(2^39) = 511 */ .quad 0x0000000000103007 /* -> level3_kernel_pgt */ #endif .org 0xc000 ENTRY(level1_ident_pgt) .fill 160,8,0 level1_ident_a0: .fill 96,8,0 /* 0xa0000 - 0xfffff */ .fill 256,8,0 .org 0xd000 ENTRY(phys_table) .fill 512,8,0 .data .align 16 .globl cpu_gdt_descr cpu_gdt_descr: .word gdt_end-cpu_gdt_table gdt: .quad cpu_gdt_table #ifdef CONFIG_SMP .rept NR_CPUS-1 .word 0 .quad 0 .endr #endif ENTRY(gdt_table32) .quad 0x0000000000000000 /* This one is magic */ .quad 0x0000000000000000 /* unused */ .quad 0x00af9a000000ffff /* __KERNEL_CS */ gdt32_end: /* We need valid kernel segments for data and code in long mode too * IRET will check the segment types kkeil 2000/10/28 * Also sysret mandates a special GDT layout */ .align L1_CACHE_BYTES /* The TLS descriptors are currently at a different place compared to i386. Hopefully nobody expects them at a fixed place (Wine?) */ ENTRY(cpu_gdt_table) .quad 0x0000000000000000 /* NULL descriptor */ .quad 0x008f9a000000ffff /* __KERNEL_COMPAT32_CS */ .quad 0x00af9a000000ffff /* __KERNEL_CS */ .quad 0x00cf92000000ffff /* __KERNEL_DS */ .quad 0x00cffe000000ffff /* __USER32_CS */ .quad 0x00cff2000000ffff /* __USER_DS, __USER32_DS */ .quad 0x00affa000000ffff /* __USER_CS */ .quad 0x00cf9a000000ffff /* __KERNEL32_CS */ .quad 0,0 /* TSS */ .quad 0 /* LDT */ .quad 0,0,0 /* three TLS descriptors */ .quad 0 /* unused now */ .quad 0x00009a000000ffff /* __KERNEL16_CS - 16bit PM for S3 wakeup. */ /* base must be patched for real base address. */ gdt_end: /* asm/segment.h:GDT_ENTRIES must match this */ /* This should be a multiple of the cache line size */ /* GDTs of other CPUs: */ .fill (GDT_SIZE * NR_CPUS) - (gdt_end - cpu_gdt_table) .align L1_CACHE_BYTES ENTRY(idt_table) .rept 256 .quad 0 .quad 0 .endr |
|
From: Itsuro O. <od...@us...> - 2005-12-15 04:53:58
|
Update of /cvsroot/mkdump/minikpatch/1.0/linus/2.6.9/include/asm-x86_64 In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv11351/include/asm-x86_64 Added Files: fixmap.h page.h pgtable.h Log Message: register mini-kernel v1 update1 to CVS --- NEW FILE: page.h --- #ifndef _X86_64_PAGE_H #define _X86_64_PAGE_H #include <linux/config.h> /* PAGE_SHIFT determines the page size */ #define PAGE_SHIFT 12 #ifdef __ASSEMBLY__ #define PAGE_SIZE (0x1 << PAGE_SHIFT) #else #define PAGE_SIZE (1UL << PAGE_SHIFT) #endif #define PAGE_MASK (~(PAGE_SIZE-1)) #define PHYSICAL_PAGE_MASK (~(PAGE_SIZE-1) & (__PHYSICAL_MASK << PAGE_SHIFT)) #define THREAD_ORDER 1 #ifdef __ASSEMBLY__ #define THREAD_SIZE (1 << (PAGE_SHIFT + THREAD_ORDER)) #else #define THREAD_SIZE (1UL << (PAGE_SHIFT + THREAD_ORDER)) #endif #define CURRENT_MASK (~(THREAD_SIZE-1)) #define LARGE_PAGE_MASK (~(LARGE_PAGE_SIZE-1)) #define LARGE_PAGE_SIZE (1UL << PMD_SHIFT) #define HPAGE_SHIFT PMD_SHIFT #define HPAGE_SIZE ((1UL) << HPAGE_SHIFT) #define HPAGE_MASK (~(HPAGE_SIZE - 1)) #define HUGETLB_PAGE_ORDER (HPAGE_SHIFT - PAGE_SHIFT) #ifdef __KERNEL__ #ifndef __ASSEMBLY__ void clear_page(void *); void copy_page(void *, void *); #define clear_user_page(page, vaddr, pg) clear_page(page) #define copy_user_page(to, from, vaddr, pg) copy_page(to, from) /* * These are used to make use of C type-checking.. */ typedef struct { unsigned long pte; } pte_t; typedef struct { unsigned long pmd; } pmd_t; typedef struct { unsigned long pgd; } pgd_t; typedef struct { unsigned long pml4; } pml4_t; #define PTE_MASK PHYSICAL_PAGE_MASK typedef struct { unsigned long pgprot; } pgprot_t; extern unsigned long pte_val(pte_t); extern unsigned long pmd_val(pmd_t); extern unsigned long pgd_val(pgd_t); extern unsigned long pml4_val(pml4_t); #define pgprot_val(x) ((x).pgprot) #define __pte(x) ((pte_t) { (x) } ) #define __pmd(x) ((pmd_t) { (x) } ) #define __pgd(x) ((pgd_t) { (x) } ) #define __pml4(x) ((pml4_t) { (x) } ) #define __pgprot(x) ((pgprot_t) { (x) } ) extern unsigned long vm_stack_flags, vm_stack_flags32; extern unsigned long vm_data_default_flags, vm_data_default_flags32; extern unsigned long vm_force_exec32; #define __START_KERNEL 0xffffffff80100000UL #define __START_KERNEL_map 0xffffffff80000000UL #define __PAGE_OFFSET 0x0000010000000000UL /* 1 << 40 */ #else #define __START_KERNEL 0xffffffff80100000 #define __START_KERNEL_map 0xffffffff80000000 #define __PAGE_OFFSET 0x0000010000000000 /* 1 << 40 */ #endif /* !__ASSEMBLY__ */ /* to align the pointer to the (next) page boundary */ #define PAGE_ALIGN(addr) (((addr)+PAGE_SIZE-1)&PAGE_MASK) /* See Documentation/x86_64/mm.txt for a description of the memory map. */ #define __PHYSICAL_MASK_SHIFT 40 #define __PHYSICAL_MASK ((1UL << __PHYSICAL_MASK_SHIFT) - 1) #define __VIRTUAL_MASK_SHIFT 48 #define __VIRTUAL_MASK ((1UL << __VIRTUAL_MASK_SHIFT) - 1) #define KERNEL_TEXT_SIZE (40UL*1024*1024) #define KERNEL_TEXT_START 0xffffffff80000000UL #ifndef __ASSEMBLY__ #include <asm/bug.h> /* Pure 2^n version of get_order */ extern __inline__ int get_order(unsigned long size) { int order; size = (size-1) >> (PAGE_SHIFT-1); order = -1; do { size >>= 1; order++; } while (size); return order; } #endif /* __ASSEMBLY__ */ #define PAGE_OFFSET ((unsigned long)__PAGE_OFFSET) /* Note: __pa(&symbol_visible_to_c) should be always replaced with __pa_symbol. Otherwise you risk miscompilation. */ #define __pa(x) (((unsigned long)(x)>=__START_KERNEL_map)?(unsigned long)(x) - (unsigned long)__START_KERNEL_map:(unsigned long)(x) - PAGE_OFFSET) /* __pa_symbol should be used for C visible symbols. This seems to be the official gcc blessed way to do such arithmetic. */ #define __pa_symbol(x) \ ({unsigned long v; \ asm("" : "=r" (v) : "0" (x)); \ __pa(v); }) #define __va(x) ((void *)((unsigned long)(x)+PAGE_OFFSET)) #ifndef CONFIG_DISCONTIGMEM #define pfn_to_page(pfn) (mem_map + (pfn)) #define page_to_pfn(page) ((unsigned long)((page) - mem_map)) #define pfn_valid(pfn) ((pfn) < max_mapnr) #endif #define virt_to_page(kaddr) pfn_to_page(__pa(kaddr) >> PAGE_SHIFT) #define virt_addr_valid(kaddr) pfn_valid(__pa(kaddr) >> PAGE_SHIFT) #define pfn_to_kaddr(pfn) __va((pfn) << PAGE_SHIFT) #define __VM_DATA_DEFAULT_FLAGS (VM_READ | VM_WRITE | VM_EXEC | \ VM_MAYREAD | VM_MAYWRITE | VM_MAYEXEC) #define __VM_STACK_FLAGS (VM_GROWSDOWN | VM_READ | VM_WRITE | VM_EXEC | \ VM_MAYREAD | VM_MAYWRITE | VM_MAYEXEC) #define VM_DATA_DEFAULT_FLAGS \ (test_thread_flag(TIF_IA32) ? vm_data_default_flags32 : \ vm_data_default_flags) #define VM_STACK_DEFAULT_FLAGS \ (test_thread_flag(TIF_IA32) ? vm_stack_flags32 : vm_stack_flags) #define CONFIG_ARCH_GATE_AREA 1 #ifndef __ASSEMBLY__ struct task_struct; struct vm_area_struct *get_gate_vma(struct task_struct *tsk); int in_gate_area(struct task_struct *task, unsigned long addr); #endif #ifdef CONFIG_DUMP_MINI_KERNEL #define MINIK_MEM_SEG_OFFSET (PAGE_SIZE) #define MINIK_DUMP_HEADER_OFFSET (PAGE_SIZE * 2) #define MEM_SEG_OFFSET (PAGE_SIZE * 3) #define MAX_MINIK_MEM_SEG 32 #define __PHYS_SEG_SHIFT 22 #define __PHYS_SEG_MASK (0x3fffffUL) #ifndef __ASSEMBLY__ extern unsigned long phys_table[]; extern unsigned long true_phys(unsigned long pa); extern unsigned long pseudo_phys(unsigned long pa); #endif #endif #endif /* __KERNEL__ */ #endif /* _X86_64_PAGE_H */ --- NEW FILE: pgtable.h --- #ifndef _X86_64_PGTABLE_H #define _X86_64_PGTABLE_H /* * This file contains the functions and defines necessary to modify and use * the x86-64 page table tree. * * x86-64 has a 4 level table setup. Generic linux MM only supports * three levels. The fourth level is currently a single static page that * is shared by everybody and just contains a pointer to the current * three level page setup on the beginning and some kernel mappings at * the end. For more details see Documentation/x86_64/mm.txt */ #include <asm/processor.h> #include <asm/fixmap.h> #include <asm/bitops.h> #include <linux/threads.h> #include <asm/pda.h> extern pgd_t level3_kernel_pgt[512]; extern pgd_t level3_physmem_pgt[512]; extern pgd_t level3_ident_pgt[512]; extern pmd_t level2_kernel_pgt[512]; extern pml4_t init_level4_pgt[]; extern pte_t level1_ident_pgt[]; extern pgd_t boot_vmalloc_pgt[]; extern unsigned long __supported_pte_mask; #define swapper_pg_dir NULL extern void paging_init(void); extern void clear_kernel_mapping(unsigned long addr, unsigned long size); extern unsigned long pgkern_mask; /* * ZERO_PAGE is a global shared page that is always zero: used * for zero-mapped memory areas etc.. */ extern unsigned long empty_zero_page[PAGE_SIZE/sizeof(unsigned long)]; #define ZERO_PAGE(vaddr) (virt_to_page(empty_zero_page)) #define PML4_SHIFT 39 #define PTRS_PER_PML4 512 /* * PGDIR_SHIFT determines what a top-level page table entry can map */ #define PGDIR_SHIFT 30 #define PTRS_PER_PGD 512 /* * PMD_SHIFT determines the size of the area a middle-level * page table can map */ #define PMD_SHIFT 21 #define PTRS_PER_PMD 512 /* * entries per page directory level */ #define PTRS_PER_PTE 512 #define pte_ERROR(e) \ printk("%s:%d: bad pte %p(%016lx).\n", __FILE__, __LINE__, &(e), pte_val(e)) #define pmd_ERROR(e) \ printk("%s:%d: bad pmd %p(%016lx).\n", __FILE__, __LINE__, &(e), pmd_val(e)) #define pgd_ERROR(e) \ printk("%s:%d: bad pgd %p(%016lx).\n", __FILE__, __LINE__, &(e), pgd_val(e)) #define pml4_none(x) (!pml4_val(x)) #define pgd_none(x) (!pgd_val(x)) extern inline int pgd_present(pgd_t pgd) { return !pgd_none(pgd); } extern void set_pte(pte_t *dst, pte_t val); extern void set_pmd(pmd_t *dst, pmd_t val); extern void set_pgd(pgd_t *dst, pgd_t val); extern inline void pgd_clear (pgd_t * pgd) { set_pgd(pgd, __pgd(0)); } extern void set_pml4(pml4_t *dst, pml4_t val); #define pgd_page(pgd) \ ((unsigned long) __va(pgd_val(pgd) & PHYSICAL_PAGE_MASK)) #define ptep_get_and_clear(xp) __pte(xchg(&(xp)->pte, 0)) #define pte_same(a, b) ((a).pte == (b).pte) #define PML4_SIZE (1UL << PML4_SHIFT) #define PML4_MASK (~(PML4_SIZE-1)) #define PMD_SIZE (1UL << PMD_SHIFT) #define PMD_MASK (~(PMD_SIZE-1)) #define PGDIR_SIZE (1UL << PGDIR_SHIFT) #define PGDIR_MASK (~(PGDIR_SIZE-1)) #define USER_PTRS_PER_PGD (TASK_SIZE/PGDIR_SIZE) #define FIRST_USER_PGD_NR 0 #define USER_PGD_PTRS (PAGE_OFFSET >> PGDIR_SHIFT) #define KERNEL_PGD_PTRS (PTRS_PER_PGD-USER_PGD_PTRS) #define TWOLEVEL_PGDIR_SHIFT 20 #define BOOT_USER_L4_PTRS 1 #define BOOT_KERNEL_L4_PTRS 511 /* But we will do it in 4rd level */ #ifndef __ASSEMBLY__ #define VMALLOC_START 0xffffff0000000000UL #define VMALLOC_END 0xffffff7fffffffffUL #define MODULES_VADDR 0xffffffffa0000000UL #define MODULES_END 0xffffffffafffffffUL #define MODULES_LEN (MODULES_END - MODULES_VADDR) #define IOMAP_START 0xfffffe8000000000UL #define _PAGE_BIT_PRESENT 0 #define _PAGE_BIT_RW 1 #define _PAGE_BIT_USER 2 #define _PAGE_BIT_PWT 3 #define _PAGE_BIT_PCD 4 #define _PAGE_BIT_ACCESSED 5 #define _PAGE_BIT_DIRTY 6 #define _PAGE_BIT_PSE 7 /* 4 MB (or 2MB) page */ #define _PAGE_BIT_GLOBAL 8 /* Global TLB entry PPro+ */ #define _PAGE_BIT_NX 63 /* No execute: only valid after cpuid check */ #define _PAGE_PRESENT 0x001 #define _PAGE_RW 0x002 #define _PAGE_USER 0x004 #define _PAGE_PWT 0x008 #define _PAGE_PCD 0x010 #define _PAGE_ACCESSED 0x020 #define _PAGE_DIRTY 0x040 #define _PAGE_PSE 0x080 /* 2MB page */ #define _PAGE_FILE 0x040 /* set:pagecache, unset:swap */ #define _PAGE_GLOBAL 0x100 /* Global TLB entry */ #define _PAGE_PROTNONE 0x080 /* If not present */ #define _PAGE_NX (1UL<<_PAGE_BIT_NX) #define _PGTABLE_FLAGS (0xfff0000000000fffUL) /* flags field of page tables */ #define _PAGE_TABLE (_PAGE_PRESENT | _PAGE_RW | _PAGE_USER | _PAGE_ACCESSED | _PAGE_DIRTY) #define _KERNPG_TABLE (_PAGE_PRESENT | _PAGE_RW | _PAGE_ACCESSED | _PAGE_DIRTY) #define _PAGE_CHG_MASK (PTE_MASK | _PAGE_ACCESSED | _PAGE_DIRTY) #define PAGE_NONE __pgprot(_PAGE_PROTNONE | _PAGE_ACCESSED) #define PAGE_SHARED __pgprot(_PAGE_PRESENT | _PAGE_RW | _PAGE_USER | _PAGE_ACCESSED | _PAGE_NX) #define PAGE_SHARED_EXEC __pgprot(_PAGE_PRESENT | _PAGE_RW | _PAGE_USER | _PAGE_ACCESSED) #define PAGE_COPY_NOEXEC __pgprot(_PAGE_PRESENT | _PAGE_USER | _PAGE_ACCESSED | _PAGE_NX) #define PAGE_COPY PAGE_COPY_NOEXEC #define PAGE_COPY_EXEC __pgprot(_PAGE_PRESENT | _PAGE_USER | _PAGE_ACCESSED) #define PAGE_READONLY __pgprot(_PAGE_PRESENT | _PAGE_USER | _PAGE_ACCESSED | _PAGE_NX) #define PAGE_READONLY_EXEC __pgprot(_PAGE_PRESENT | _PAGE_USER | _PAGE_ACCESSED) #define __PAGE_KERNEL \ (_PAGE_PRESENT | _PAGE_RW | _PAGE_DIRTY | _PAGE_ACCESSED | _PAGE_NX) #define __PAGE_KERNEL_EXEC \ (_PAGE_PRESENT | _PAGE_RW | _PAGE_DIRTY | _PAGE_ACCESSED) #define __PAGE_KERNEL_NOCACHE \ (_PAGE_PRESENT | _PAGE_RW | _PAGE_DIRTY | _PAGE_PCD | _PAGE_ACCESSED | _PAGE_NX) #define __PAGE_KERNEL_RO \ (_PAGE_PRESENT | _PAGE_DIRTY | _PAGE_ACCESSED | _PAGE_NX) #define __PAGE_KERNEL_VSYSCALL \ (_PAGE_PRESENT | _PAGE_USER | _PAGE_ACCESSED) #define __PAGE_KERNEL_VSYSCALL_NOCACHE \ (_PAGE_PRESENT | _PAGE_USER | _PAGE_ACCESSED | _PAGE_PCD) #define __PAGE_KERNEL_LARGE \ (__PAGE_KERNEL | _PAGE_PSE) #define MAKE_GLOBAL(x) __pgprot((x) | _PAGE_GLOBAL) #define PAGE_KERNEL MAKE_GLOBAL(__PAGE_KERNEL) #define PAGE_KERNEL_EXEC MAKE_GLOBAL(__PAGE_KERNEL_EXEC) #define PAGE_KERNEL_RO MAKE_GLOBAL(__PAGE_KERNEL_RO) #define PAGE_KERNEL_NOCACHE MAKE_GLOBAL(__PAGE_KERNEL_NOCACHE) #define PAGE_KERNEL_VSYSCALL MAKE_GLOBAL(__PAGE_KERNEL_VSYSCALL) #define PAGE_KERNEL_LARGE MAKE_GLOBAL(__PAGE_KERNEL_LARGE) #define PAGE_KERNEL_VSYSCALL_NOCACHE MAKE_GLOBAL(__PAGE_KERNEL_VSYSCALL_NOCACHE) /* xwr */ #define __P000 PAGE_NONE #define __P001 PAGE_READONLY #define __P010 PAGE_COPY #define __P011 PAGE_COPY #define __P100 PAGE_READONLY_EXEC #define __P101 PAGE_READONLY_EXEC #define __P110 PAGE_COPY_EXEC #define __P111 PAGE_COPY_EXEC #define __S000 PAGE_NONE #define __S001 PAGE_READONLY #define __S010 PAGE_SHARED #define __S011 PAGE_SHARED #define __S100 PAGE_READONLY_EXEC #define __S101 PAGE_READONLY_EXEC #define __S110 PAGE_SHARED_EXEC #define __S111 PAGE_SHARED_EXEC static inline unsigned long pgd_bad(pgd_t pgd) { unsigned long val = pgd_val(pgd); val &= ~PTE_MASK; val &= ~(_PAGE_USER | _PAGE_DIRTY); return val & ~(_PAGE_PRESENT | _PAGE_RW | _PAGE_ACCESSED); } #define pte_none(x) (!pte_val(x)) #define pte_present(x) (pte_val(x) & (_PAGE_PRESENT | _PAGE_PROTNONE)) #define pte_clear(xp) do { set_pte(xp, __pte(0)); } while (0) #define pages_to_mb(x) ((x) >> (20-PAGE_SHIFT)) /* FIXME: is this right? */ #define pte_page(x) pfn_to_page(pte_pfn(x)) #define pte_pfn(x) ((pte_val(x) >> PAGE_SHIFT) & __PHYSICAL_MASK) static inline pte_t pfn_pte(unsigned long page_nr, pgprot_t pgprot) { pte_t pte; pte.pte = (page_nr << PAGE_SHIFT); pte.pte |= pgprot_val(pgprot); pte.pte &= __supported_pte_mask; return pte; } /* * The following only work if pte_present() is true. * Undefined behaviour if not.. */ static inline int pte_user(pte_t pte) { return pte_val(pte) & _PAGE_USER; } extern inline int pte_read(pte_t pte) { return pte_val(pte) & _PAGE_USER; } extern inline int pte_exec(pte_t pte) { return pte_val(pte) & _PAGE_USER; } extern inline int pte_dirty(pte_t pte) { return pte_val(pte) & _PAGE_DIRTY; } extern inline int pte_young(pte_t pte) { return pte_val(pte) & _PAGE_ACCESSED; } extern inline int pte_write(pte_t pte) { return pte_val(pte) & _PAGE_RW; } static inline int pte_file(pte_t pte) { return pte_val(pte) & _PAGE_FILE; } extern inline pte_t pte_rdprotect(pte_t pte) { set_pte(&pte, __pte(pte_val(pte) & ~_PAGE_USER)); return pte; } extern inline pte_t pte_exprotect(pte_t pte) { set_pte(&pte, __pte(pte_val(pte) & ~_PAGE_USER)); return pte; } extern inline pte_t pte_mkclean(pte_t pte) { set_pte(&pte, __pte(pte_val(pte) & ~_PAGE_DIRTY)); return pte; } extern inline pte_t pte_mkold(pte_t pte) { set_pte(&pte, __pte(pte_val(pte) & ~_PAGE_ACCESSED)); return pte; } extern inline pte_t pte_wrprotect(pte_t pte) { set_pte(&pte, __pte(pte_val(pte) & ~_PAGE_RW)); return pte; } extern inline pte_t pte_mkread(pte_t pte) { set_pte(&pte, __pte(pte_val(pte) | _PAGE_USER)); return pte; } extern inline pte_t pte_mkexec(pte_t pte) { set_pte(&pte, __pte(pte_val(pte) | _PAGE_USER)); return pte; } extern inline pte_t pte_mkdirty(pte_t pte) { set_pte(&pte, __pte(pte_val(pte) | _PAGE_DIRTY)); return pte; } extern inline pte_t pte_mkyoung(pte_t pte) { set_pte(&pte, __pte(pte_val(pte) | _PAGE_ACCESSED)); return pte; } extern inline pte_t pte_mkwrite(pte_t pte) { set_pte(&pte, __pte(pte_val(pte) | _PAGE_RW)); return pte; } static inline int ptep_test_and_clear_dirty(pte_t *ptep) { if (!pte_dirty(*ptep)) return 0; return test_and_clear_bit(_PAGE_BIT_DIRTY, ptep); } static inline int ptep_test_and_clear_young(pte_t *ptep) { if (!pte_young(*ptep)) return 0; return test_and_clear_bit(_PAGE_BIT_ACCESSED, ptep); } static inline void ptep_set_wrprotect(pte_t *ptep) { clear_bit(_PAGE_BIT_RW, ptep); } static inline void ptep_mkdirty(pte_t *ptep) { set_bit(_PAGE_BIT_DIRTY, ptep); } /* * Macro to mark a page protection value as "uncacheable". */ #define pgprot_noncached(prot) (__pgprot(pgprot_val(prot) | _PAGE_PCD | _PAGE_PWT)) #define __LARGE_PTE (_PAGE_PSE|_PAGE_PRESENT) static inline int pmd_large(pmd_t pte) { return (pmd_val(pte) & __LARGE_PTE) == __LARGE_PTE; } /* * Conversion functions: convert a page and protection to a page entry, * and a page entry and page directory to the page they refer to. */ #define page_pte(page) page_pte_prot(page, __pgprot(0)) /* * Level 4 access. * Never use these in the common code. */ #define pml4_page(pml4) ((unsigned long) __va(pml4_val(pml4) & PTE_MASK)) #define pml4_index(address) ((address >> PML4_SHIFT) & (PTRS_PER_PML4-1)) #define pml4_offset_k(address) (init_level4_pgt + pml4_index(address)) #define pml4_present(pml4) (pml4_val(pml4) & _PAGE_PRESENT) #define mk_kernel_pml4(address) ((pml4_t){ (address) | _KERNPG_TABLE }) #define level3_offset_k(dir, address) ((pgd_t *) pml4_page(*(dir)) + pgd_index(address)) /* PGD - Level3 access */ /* to find an entry in a page-table-directory. */ #define pgd_index(address) ((address >> PGDIR_SHIFT) & (PTRS_PER_PGD-1)) static inline pgd_t *__pgd_offset_k(pgd_t *pgd, unsigned long address) { return pgd + pgd_index(address); } /* Find correct pgd via the hidden fourth level page level: */ /* This accesses the reference page table of the boot cpu. Other CPUs get synced lazily via the page fault handler. */ static inline pgd_t *pgd_offset_k(unsigned long address) { unsigned long addr; addr = pml4_val(init_level4_pgt[pml4_index(address)]); addr &= PHYSICAL_PAGE_MASK; return __pgd_offset_k((pgd_t *)__va(addr), address); } /* Access the pgd of the page table as seen by the current CPU. */ static inline pgd_t *current_pgd_offset_k(unsigned long address) { unsigned long addr; addr = read_pda(level4_pgt)[pml4_index(address)]; addr &= PHYSICAL_PAGE_MASK; return __pgd_offset_k((pgd_t *)__va(addr), address); } #define pgd_offset(mm, address) ((mm)->pgd+pgd_index(address)) /* PMD - Level 2 access */ #define pmd_page_kernel(pmd) ((unsigned long) __va(pmd_val(pmd) & PTE_MASK)) #define pmd_page(pmd) (pfn_to_page(pmd_val(pmd) >> PAGE_SHIFT)) #define pmd_index(address) (((address) >> PMD_SHIFT) & (PTRS_PER_PMD-1)) #define pmd_offset(dir, address) ((pmd_t *) pgd_page(*(dir)) + \ pmd_index(address)) #define pmd_none(x) (!pmd_val(x)) #define pmd_present(x) (pmd_val(x) & _PAGE_PRESENT) #define pmd_clear(xp) do { set_pmd(xp, __pmd(0)); } while (0) #define pmd_bad(x) ((pmd_val(x) & (~PTE_MASK & ~_PAGE_USER)) != _KERNPG_TABLE ) #define pfn_pmd(nr,prot) (__pmd(((nr) << PAGE_SHIFT) | pgprot_val(prot))) #define pmd_pfn(x) ((pmd_val(x) >> PAGE_SHIFT) & __PHYSICAL_MASK) #define pte_to_pgoff(pte) ((pte_val(pte) & PHYSICAL_PAGE_MASK) >> PAGE_SHIFT) #define pgoff_to_pte(off) ((pte_t) { ((off) << PAGE_SHIFT) | _PAGE_FILE }) #define PTE_FILE_MAX_BITS __PHYSICAL_MASK_SHIFT /* PTE - Level 1 access. */ /* page, protection -> pte */ #define mk_pte(page, pgprot) pfn_pte(page_to_pfn(page), (pgprot)) #define mk_pte_huge(entry) ((entry).pte |= _PAGE_PRESENT | _PAGE_PSE) /* physical address -> PTE */ static inline pte_t mk_pte_phys(unsigned long physpage, pgprot_t pgprot) { pte_t pte; pte.pte = physpage | pgprot_val(pgprot); return pte; } /* Change flags of a PTE */ extern inline pte_t pte_modify(pte_t pte, pgprot_t newprot) { pte.pte &= _PAGE_CHG_MASK; pte.pte |= pgprot_val(newprot); pte.pte &= __supported_pte_mask; return pte; } #define pte_index(address) \ ((address >> PAGE_SHIFT) & (PTRS_PER_PTE - 1)) #define pte_offset_kernel(dir, address) ((pte_t *) pmd_page_kernel(*(dir)) + \ pte_index(address)) /* x86-64 always has all page tables mapped. */ #define pte_offset_map(dir,address) pte_offset_kernel(dir,address) #define pte_offset_map_nested(dir,address) pte_offset_kernel(dir,address) #define pte_unmap(pte) /* NOP */ #define pte_unmap_nested(pte) /* NOP */ #define update_mmu_cache(vma,address,pte) do { } while (0) /* We only update the dirty/accessed state if we set * the dirty bit by hand in the kernel, since the hardware * will do the accessed bit for us, and we don't want to * race with other CPU's that might be updating the dirty * bit at the same time. */ #define __HAVE_ARCH_PTEP_SET_ACCESS_FLAGS #define ptep_set_access_flags(__vma, __address, __ptep, __entry, __dirty) \ do { \ if (__dirty) { \ set_pte(__ptep, __entry); \ flush_tlb_page(__vma, __address); \ } \ } while (0) /* Encode and de-code a swap entry */ #define __swp_type(x) (((x).val >> 1) & 0x3f) #define __swp_offset(x) ((x).val >> 8) #define __swp_entry(type, offset) ((swp_entry_t) { ((type) << 1) | ((offset) << 8) }) #define __pte_to_swp_entry(pte) ((swp_entry_t) { pte_val(pte) }) #define __swp_entry_to_pte(x) ((pte_t) { (x).val }) #endif /* !__ASSEMBLY__ */ extern int kern_addr_valid(unsigned long addr); #define io_remap_page_range remap_page_range #define HAVE_ARCH_UNMAPPED_AREA #define pgtable_cache_init() do { } while (0) #define check_pgt_cache() do { } while (0) #define PAGE_AGP PAGE_KERNEL_NOCACHE #define HAVE_PAGE_AGP 1 /* fs/proc/kcore.c */ #define kc_vaddr_to_offset(v) ((v) & __VIRTUAL_MASK) #define kc_offset_to_vaddr(o) \ (((o) & (1UL << (__VIRTUAL_MASK_SHIFT-1))) ? ((o) | (~__VIRTUAL_MASK)) : (o)) #define __HAVE_ARCH_PTEP_TEST_AND_CLEAR_YOUNG #define __HAVE_ARCH_PTEP_TEST_AND_CLEAR_DIRTY #define __HAVE_ARCH_PTEP_GET_AND_CLEAR #define __HAVE_ARCH_PTEP_SET_WRPROTECT #define __HAVE_ARCH_PTEP_MKDIRTY #define __HAVE_ARCH_PTE_SAME #include <asm-generic/pgtable.h> #endif /* _X86_64_PGTABLE_H */ --- NEW FILE: fixmap.h --- /* * fixmap.h: compile-time virtual memory allocation * * This file is subject to the terms and conditions of the GNU General Public * License. See the file "COPYING" in the main directory of this archive * for more details. * * Copyright (C) 1998 Ingo Molnar */ #ifndef _ASM_FIXMAP_H #define _ASM_FIXMAP_H #include <linux/config.h> #include <linux/kernel.h> #include <asm/apicdef.h> #include <asm/page.h> #include <asm/vsyscall.h> #include <asm/vsyscall32.h> /* * Here we define all the compile-time 'special' virtual * addresses. The point is to have a constant address at * compile time, but to set the physical address only * in the boot process. * * these 'compile-time allocated' memory buffers are * fixed-size 4k pages. (or larger if used with an increment * highger than 1) use fixmap_set(idx,phys) to associate * physical memory with fixmap indices. * * TLB entries of such buffers will not be flushed across * task switches. */ enum fixed_addresses { VSYSCALL_LAST_PAGE, VSYSCALL_FIRST_PAGE = VSYSCALL_LAST_PAGE + ((VSYSCALL_END-VSYSCALL_START) >> PAGE_SHIFT) - 1, VSYSCALL_HPET, FIX_HPET_BASE, #ifdef CONFIG_X86_LOCAL_APIC FIX_APIC_BASE, /* local (CPU) APIC) -- required for SMP or not */ #endif #ifdef CONFIG_X86_IO_APIC FIX_IO_APIC_BASE_0, FIX_IO_APIC_BASE_END = FIX_IO_APIC_BASE_0 + MAX_IO_APICS-1, #endif FIX_DUMP, __end_of_fixed_addresses }; extern void __set_fixmap (enum fixed_addresses idx, unsigned long phys, pgprot_t flags); #define set_fixmap(idx, phys) \ __set_fixmap(idx, phys, PAGE_KERNEL) /* * Some hardware wants to get fixmapped without caching. */ #define set_fixmap_nocache(idx, phys) \ __set_fixmap(idx, phys, PAGE_KERNEL_NOCACHE) #define FIXADDR_TOP (VSYSCALL_END-PAGE_SIZE) #define FIXADDR_SIZE (__end_of_fixed_addresses << PAGE_SHIFT) #define FIXADDR_START (FIXADDR_TOP - FIXADDR_SIZE) /* Only covers 32bit vsyscalls currently. Need another set for 64bit. */ #define FIXADDR_USER_START ((unsigned long)VSYSCALL32_VSYSCALL) #define FIXADDR_USER_END (FIXADDR_USER_START + PAGE_SIZE) #define __fix_to_virt(x) (FIXADDR_TOP - ((x) << PAGE_SHIFT)) extern void __this_fixmap_does_not_exist(void); /* * 'index to address' translation. If anyone tries to use the idx * directly without translation, we catch the bug with a NULL-deference * kernel oops. Illegal ranges of incoming indices are caught too. */ extern inline unsigned long fix_to_virt(const unsigned int idx) { /* * this branch gets completely eliminated after inlining, * except when someone tries to use fixaddr indices in an * illegal way. (such as mixing up address types or using * out-of-range indices). * * If it doesn't get removed, the linker will complain * loudly with a reasonably clear error message.. */ if (idx >= __end_of_fixed_addresses) __this_fixmap_does_not_exist(); return __fix_to_virt(idx); } #endif |
|
From: Itsuro O. <od...@us...> - 2005-12-15 04:53:58
|
Update of /cvsroot/mkdump/minikpatch/1.0/linus/2.6.9/arch/i386/kernel In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv11351/arch/i386/kernel Added Files: head.S irq.c pci-dma.c setup.c Log Message: register mini-kernel v1 update1 to CVS --- NEW FILE: pci-dma.c --- /* * Dynamic DMA mapping support. * * On i386 there is no hardware dynamic DMA address translation, * so consistent alloc/free are merely page allocation/freeing. * The rest of the dynamic DMA mapping interface is implemented * in asm/pci.h. */ #include <linux/types.h> #include <linux/mm.h> #include <linux/string.h> #include <linux/pci.h> #include <asm/io.h> struct dma_coherent_mem { void *virt_base; u32 device_base; int size; int flags; unsigned long *bitmap; }; void *dma_alloc_coherent(struct device *dev, size_t size, dma_addr_t *dma_handle, int gfp) { void *ret; struct dma_coherent_mem *mem = dev ? dev->dma_mem : NULL; int order = get_order(size); /* ignore region specifiers */ gfp &= ~(__GFP_DMA | __GFP_HIGHMEM); if (mem) { int page = bitmap_find_free_region(mem->bitmap, mem->size, order); if (page >= 0) { *dma_handle = mem->device_base + (page << PAGE_SHIFT); ret = mem->virt_base + (page << PAGE_SHIFT); memset(ret, 0, size); return ret; } if (mem->flags & DMA_MEMORY_EXCLUSIVE) return NULL; } if (dev == NULL || (dev->coherent_dma_mask < 0xffffffff)) gfp |= GFP_DMA; ret = (void *)__get_free_pages(gfp, order); if (ret != NULL) { memset(ret, 0, size); *dma_handle = true_phys(virt_to_phys(ret)); } return ret; } void dma_free_coherent(struct device *dev, size_t size, void *vaddr, dma_addr_t dma_handle) { struct dma_coherent_mem *mem = dev ? dev->dma_mem : NULL; int order = get_order(size); if (mem && vaddr >= mem->virt_base && vaddr < (mem->virt_base + (mem->size << PAGE_SHIFT))) { int page = (vaddr - mem->virt_base) >> PAGE_SHIFT; bitmap_release_region(mem->bitmap, page, order); } else free_pages((unsigned long)vaddr, order); } int dma_declare_coherent_memory(struct device *dev, dma_addr_t bus_addr, dma_addr_t device_addr, size_t size, int flags) { void __iomem *mem_base; int pages = size >> PAGE_SHIFT; int bitmap_size = (pages + 31)/32; if ((flags & (DMA_MEMORY_MAP | DMA_MEMORY_IO)) == 0) goto out; if (!size) goto out; if (dev->dma_mem) goto out; /* FIXME: this routine just ignores DMA_MEMORY_INCLUDES_CHILDREN */ mem_base = ioremap(bus_addr, size); if (!mem_base) goto out; dev->dma_mem = kmalloc(GFP_KERNEL, sizeof(struct dma_coherent_mem)); if (!dev->dma_mem) goto out; memset(dev->dma_mem, 0, sizeof(struct dma_coherent_mem)); dev->dma_mem->bitmap = kmalloc(GFP_KERNEL, bitmap_size); if (!dev->dma_mem->bitmap) goto free1_out; memset(dev->dma_mem->bitmap, 0, bitmap_size); dev->dma_mem->virt_base = mem_base; dev->dma_mem->device_base = device_addr; dev->dma_mem->size = pages; dev->dma_mem->flags = flags; if (flags & DMA_MEMORY_MAP) return DMA_MEMORY_MAP; return DMA_MEMORY_IO; free1_out: kfree(dev->dma_mem->bitmap); out: return 0; } EXPORT_SYMBOL(dma_declare_coherent_memory); void dma_release_declared_memory(struct device *dev) { struct dma_coherent_mem *mem = dev->dma_mem; if(!mem) return; dev->dma_mem = NULL; kfree(mem->bitmap); kfree(mem); } EXPORT_SYMBOL(dma_release_declared_memory); void *dma_mark_declared_memory_occupied(struct device *dev, dma_addr_t device_addr, size_t size) { struct dma_coherent_mem *mem = dev->dma_mem; int pages = (size + (device_addr & ~PAGE_MASK) + PAGE_SIZE - 1) >> PAGE_SHIFT; int pos, err; if (!mem) return ERR_PTR(-EINVAL); pos = (device_addr - mem->device_base) >> PAGE_SHIFT; err = bitmap_allocate_region(mem->bitmap, pos, get_order(pages)); if (err != 0) return ERR_PTR(err); return mem->virt_base + (pos << PAGE_SHIFT); } EXPORT_SYMBOL(dma_mark_declared_memory_occupied); --- NEW FILE: setup.c --- /* * linux/arch/i386/kernel/setup.c * * Copyright (C) 1995 Linus Torvalds * * Support of BIGMEM added by Gerhard Wichert, Siemens AG, July 1999 * * Memory region support * David Parsons <or...@pe...>, July-August 1999 * * Added E820 sanitization routine (removes overlapping memory regions); * Brian Moyle <bm...@mv...>, February 2001 * * Moved CPU detection code to cpu/${cpu}.c * Patrick Mochel <mo...@os...>, March 2002 * * Provisions for empty E820 memory regions (reported by certain BIOSes). * Alex Achenbach <xe...@sl...>, December 2002. * [...1406 lines suppressed...] register_memory(max_low_pfn); #ifdef CONFIG_VT #if defined(CONFIG_VGA_CONSOLE) if (!efi_enabled || (efi_mem_type(0xa0000) != EFI_CONVENTIONAL_MEMORY)) conswitchp = &vga_con; #elif defined(CONFIG_DUMMY_CONSOLE) conswitchp = &dummy_con; #endif #endif } #include "setup_arch_post.h" /* * Local Variables: * mode:c * c-file-style:"k&r" * c-basic-offset:8 * End: */ --- NEW FILE: irq.c --- /* * linux/arch/i386/kernel/irq.c * * Copyright (C) 1992, 1998 Linus Torvalds, Ingo Molnar * * This file contains the code used by various IRQ handling routines: * asking for different IRQ's should be done through these routines * instead of just grabbing them. Thus setups with different IRQ numbers * shouldn't result in any weird surprises, and installing new handlers * should be easier. */ /* * (mostly architecture independent, will move to kernel/irq.c in 2.5.) * * IRQs are in fact implemented a bit like signal handlers for the kernel. * Naturally it's not a 1:1 relation, but there are similarities. */ [...1119 lines suppressed...] /* build the stack frame on the softirq stack */ isp = (u32*) ((char*)irqctx + sizeof(*irqctx)); asm volatile( " xchgl %%ebx,%%esp \n" " call __do_softirq \n" " movl %%ebx,%%esp \n" : "=b"(isp) : "0"(isp) : "memory", "cc", "edx", "ecx", "eax" ); } local_irq_restore(flags); } EXPORT_SYMBOL(do_softirq); #endif --- NEW FILE: head.S --- /* * linux/arch/i386/kernel/head.S -- the 32-bit startup code. * * Copyright (C) 1991, 1992 Linus Torvalds * * Enhanced CPU detection and feature setting code by Mike Jagdis * and Martin Mares, November 1997. */ /* * Modified for Mini Kernel * Portions Copyright 2004 NTT DATA CORPORATION. * Portions Copyright 2004 VA Linux Systems Japan K.K. */ .text #include <linux/config.h> #include <linux/threads.h> #include <linux/linkage.h> #include <asm/segment.h> #include <asm/page.h> #include <asm/pgtable.h> #include <asm/desc.h> #include <asm/cache.h> #include <asm/thread_info.h> #include <asm/asm_offsets.h> #include <asm/setup.h> /* * References to members of the new_cpu_data structure. */ #define X86 new_cpu_data+CPUINFO_x86 #define X86_VENDOR new_cpu_data+CPUINFO_x86_vendor #define X86_MODEL new_cpu_data+CPUINFO_x86_model #define X86_MASK new_cpu_data+CPUINFO_x86_mask #define X86_HARD_MATH new_cpu_data+CPUINFO_hard_math #define X86_CPUID new_cpu_data+CPUINFO_cpuid_level #define X86_CAPABILITY new_cpu_data+CPUINFO_x86_capability #define X86_VENDOR_ID new_cpu_data+CPUINFO_x86_vendor_id /* * To be sure this is not SMP */ #ifdef CONFIG_SMP #error "Mini Kernel does not support SMP" #endif /* * 32-bit kernel entrypoint; only used by the boot CPU. On entry, * %esi points to the real-mode code as a 32-bit pointer. * CS and DS must be 4 GB flat segments, but we don't depend on * any particular GDT layout, because we load our own as soon as we * can. */ ENTRY(startup_32) /* * Calculate physical address */ call 1f 1: popl %ebx subl $0x100000, %ebx andl $0xfffff000, %ebx /* * Set segments to known values. */ cld movl $boot_gdt_descr - __PAGE_OFFSET, %eax addl %ebx, %eax /* adjust TRUE physical addr */ movl 2(%eax), %ecx addl %ebx, %ecx movl %ecx, 2(%eax) /* adjust TRUE physical addr */ lgdt (%eax) movl $(__BOOT_DS),%eax movl %eax,%ds movl %eax,%es movl %eax,%fs movl %eax,%gs /* * Clear BSS first so that there are no surprises... * No need to cld as DF is already clear from cld above... */ xorl %eax,%eax movl $__bss_start - __PAGE_OFFSET,%edi movl $__bss_stop - __PAGE_OFFSET,%ecx subl %edi,%ecx addl %ebx, %edi /* adjust TRUE physical addr */ shrl $2,%ecx rep ; stosl /* * Initialize page tables. This creates a PDE and a set of page * tables, which are first 4MB of TRUE physical memory. The variable * init_pg_tables_end is set up to point to the first "safe" location. * Mappings are created both at virtual address 0 (identity mapping) * and PAGE_OFFSET and TRUE identity mapping. * * Warning: don't use %esi or the stack in this code. However, %esp * can be used as a GPR if you really need it... */ page_pde_offset = (__PAGE_OFFSET >> 20); movl $(pg0 - __PAGE_OFFSET), %edi addl %ebx, %edi /* adjust TRUE physical addr */ movl $(swapper_pg_dir - __PAGE_OFFSET), %edx addl %ebx, %edx /* adjust TRUE physical addr */ leal 0x007(%edi),%ecx /* Create PDE entry */ movl %ecx,(%edx) /* Store identity PDE entry */ movl %ecx,page_pde_offset(%edx) /* Store kernel PDE entry */ /* Store TRUE identity PDE entry */ movl %ebx, %eax shrl $20, %eax addl %eax, %edx movl %ecx,(%edx) movl $0x007, %eax /* 0x007 = PRESENT+RW+USER */ addl %ebx, %eax /* adjust TRUE physical addr */ movl $1024, %ecx 11: stosl addl $0x1000,%eax loop 11b movl $init_pg_tables_end - __PAGE_OFFSET, %eax addl %ebx, %eax /* adjust TRUE physical addr */ subl %ebx, %edi /* back to PSEUDO physical addr */ movl %edi,(%eax) /* reset address 0x0 */ movl $pg0 - __PAGE_OFFSET, %eax addl %ebx, %eax /* adjust TRUE physical addr */ movl $0x007, (%eax) /* reset address range [0xa0000-0xfffff] for use video ram */ movl $pg0 - __PAGE_OFFSET + 0x280, %edi addl %ebx, %edi /* adjust TRUE physical addr */ movl $96, %ecx /* 96 entries */ movl $0xa0007, %eax 12: stosl addl $0x1000, %eax loop 12b; #ifdef CONFIG_SMP xorl %ebx,%ebx /* This is the boot CPU (BSP) */ jmp 3f /* * Non-boot CPU entry point; entered from trampoline.S * We can't lgdt here, because lgdt itself uses a data segment, but * we know the trampoline has already loaded the boot_gdt_table GDT * for us. */ ENTRY(startup_32_smp) cld movl $(__BOOT_DS),%eax movl %eax,%ds movl %eax,%es movl %eax,%fs movl %eax,%gs xorl %ebx,%ebx incl %ebx /* This is a secondary processor (AP) */ /* * New page tables may be in 4Mbyte page mode and may * be using the global pages. * * NOTE! If we are on a 486 we may have no cr4 at all! * So we do not try to touch it unless we really have * some bits in it to set. This won't work if the BSP * implements cr4 but this AP does not -- very unlikely * but be warned! The same applies to the pse feature * if not equally supported. --macro * * NOTE! We have to correct for the fact that we're * not yet offset PAGE_OFFSET.. */ #define cr4_bits mmu_cr4_features-__PAGE_OFFSET movl cr4_bits,%edx andl %edx,%edx jz 3f movl %cr4,%eax # Turn on paging options (PSE,PAE,..) orl %edx,%eax movl %eax,%cr4 btl $5, %eax # check if PAE is enabled jnc 6f /* Check if extended functions are implemented */ movl $0x80000000, %eax cpuid cmpl $0x80000000, %eax jbe 6f mov $0x80000001, %eax cpuid /* Execute Disable bit supported? */ btl $20, %edx jnc 6f /* Setup EFER (Extended Feature Enable Register) */ movl $0xc0000080, %ecx rdmsr btsl $11, %eax /* Make changes effective */ wrmsr 6: /* cpuid clobbered ebx, set it up again: */ xorl %ebx,%ebx incl %ebx 3: #endif /* CONFIG_SMP */ /* * Enable paging */ movl $swapper_pg_dir-__PAGE_OFFSET,%eax addl %ebx, %eax /* adjust TRUE physical addr */ movl %eax,%cr3 /* set the page table pointer.. */ movl %cr0,%eax orl $0x80000000,%eax movl %eax,%cr0 /* ..and set paging (PG) bit */ ljmp $__BOOT_CS,$1f /* Clear prefetch and normalize %eip */ 1: /* Set __phys_offset for later use */ movl %ebx, (__phys_offset) xorl %ebx, %ebx /* nen no tame */ /* Set up the stack pointer */ lss stack_start,%esp /* * Initialize eflags. Some BIOS's leave bits like NT set. This would * confuse the debugger if this code is traced. * XXX - best to initialize before switching to protected mode. */ pushl $0 popfl #ifdef CONFIG_SMP andl %ebx,%ebx jz 1f /* Initial CPU cleans BSS */ jmp checkCPUtype 1: #endif /* CONFIG_SMP */ /* * start system 32-bit setup. We need to re-do some of the things done * in 16-bit mode for the "real" operations. */ call setup_idt /* * Copy bootup parameters out of the way. * Note: %esi still has the pointer to the real-mode data. */ movl $boot_params,%edi movl $(PARAM_SIZE/4),%ecx cld rep movsl movl boot_params+NEW_CL_POINTER,%esi andl %esi,%esi jnz 2f # New command line protocol cmpw $(OLD_CL_MAGIC),OLD_CL_MAGIC_ADDR jne 1f movzwl OLD_CL_OFFSET,%esi addl $(OLD_CL_BASE_ADDR),%esi 2: movl $saved_command_line,%edi movl $(COMMAND_LINE_SIZE/4),%ecx rep movsl 1: checkCPUtype: movl $-1,X86_CPUID # -1 for no CPUID initially /* check if it is 486 or 386. */ /* * XXX - this does a lot of unnecessary setup. Alignment checks don't * apply at our cpl of 0 and the stack ought to be aligned already, and * we don't need to preserve eflags. */ movb $3,X86 # at least 386 pushfl # push EFLAGS popl %eax # get EFLAGS movl %eax,%ecx # save original EFLAGS xorl $0x240000,%eax # flip AC and ID bits in EFLAGS pushl %eax # copy to EFLAGS popfl # set EFLAGS pushfl # get new EFLAGS popl %eax # put it in eax xorl %ecx,%eax # change in flags pushl %ecx # restore original EFLAGS popfl testl $0x40000,%eax # check if AC bit changed je is386 movb $4,X86 # at least 486 testl $0x200000,%eax # check if ID bit changed je is486 /* get vendor info */ xorl %eax,%eax # call CPUID with 0 -> return vendor ID cpuid movl %eax,X86_CPUID # save CPUID level movl %ebx,X86_VENDOR_ID # lo 4 chars movl %edx,X86_VENDOR_ID+4 # next 4 chars movl %ecx,X86_VENDOR_ID+8 # last 4 chars orl %eax,%eax # do we have processor info as well? je is486 movl $1,%eax # Use the CPUID instruction to get CPU type cpuid movb %al,%cl # save reg for future use andb $0x0f,%ah # mask processor family movb %ah,X86 andb $0xf0,%al # mask model shrb $4,%al movb %al,X86_MODEL andb $0x0f,%cl # mask mask revision movb %cl,X86_MASK movl %edx,X86_CAPABILITY is486: movl $0x50022,%ecx # set AM, WP, NE and MP jmp 2f is386: movl $2,%ecx # set MP 2: movl %cr0,%eax andl $0x80000011,%eax # Save PG,PE,ET orl %ecx,%eax movl %eax,%cr0 call check_x87 incb ready lgdt cpu_gdt_descr lidt idt_descr ljmp $(__KERNEL_CS),$1f 1: movl $(__KERNEL_DS),%eax # reload all the segment registers movl %eax,%ss # after changing gdt. movl $(__USER_DS),%eax # DS/ES contains default USER segment movl %eax,%ds movl %eax,%es xorl %eax,%eax # Clear FS/GS and LDT movl %eax,%fs movl %eax,%gs lldt %ax cld # gcc2 wants the direction flag cleared at all times #ifdef CONFIG_SMP movb ready, %cl cmpb $1,%cl je 1f # the first CPU calls start_kernel # all other CPUs call initialize_secondary call initialize_secondary jmp L6 1: #endif /* CONFIG_SMP */ call start_kernel L6: jmp L6 # main should never return here, but # just in case, we know what happens. /* * We depend on ET to be correct. This checks for 287/387. */ check_x87: movb $0,X86_HARD_MATH clts fninit fstsw %ax cmpb $0,%al je 1f movl %cr0,%eax /* no coprocessor: have to set bits */ xorl $4,%eax /* set EM */ movl %eax,%cr0 ret ALIGN 1: movb $1,X86_HARD_MATH .byte 0xDB,0xE4 /* fsetpm for 287, ignored by 387 */ ret /* * setup_idt * * sets up a idt with 256 entries pointing to * ignore_int, interrupt gates. It doesn't actually load * idt - that can be done only after paging has been enabled * and the kernel moved to PAGE_OFFSET. Interrupts * are enabled elsewhere, when we can be relatively * sure everything is ok. * * Warning: %esi is live across this function. */ setup_idt: lea ignore_int,%edx movl $(__KERNEL_CS << 16),%eax movw %dx,%ax /* selector = 0x0010 = cs */ movw $0x8E00,%dx /* interrupt gate - dpl=0, present */ lea idt_table,%edi mov $256,%ecx rp_sidt: movl %eax,(%edi) movl %edx,4(%edi) addl $8,%edi dec %ecx jne rp_sidt ret /* This is the default interrupt "handler" :-) */ ALIGN ignore_int: cld pushl %eax pushl %ecx pushl %edx pushl %es pushl %ds movl $(__KERNEL_DS),%eax movl %eax,%ds movl %eax,%es pushl 16(%esp) pushl 24(%esp) pushl 32(%esp) pushl 40(%esp) pushl $int_msg call printk addl $(5*4),%esp popl %ds popl %es popl %edx popl %ecx popl %eax iret /* * Real beginning of normal "text" segment */ ENTRY(stext) ENTRY(_stext) /* * BSS section */ .section ".bss.page_aligned","w" ENTRY(swapper_pg_dir) .fill 1024,4,0 ENTRY(empty_zero_page) .fill 4096,1,0 /* * This starts the data section. */ .data ENTRY(stack_start) .long init_thread_union+THREAD_SIZE .long __BOOT_DS ready: .byte 0 int_msg: .asciz "Unknown interrupt or fault at EIP %p %p %p\n" /* * The IDT and GDT 'descriptors' are a strange 48-bit object * only used by the lidt and lgdt instructions. They are not * like usual segment descriptors - they consist of a 16-bit * segment size, and 32-bit linear address value: */ .globl boot_gdt_descr .globl idt_descr .globl cpu_gdt_descr ALIGN # early boot GDT descriptor (must use 1:1 address mapping) .word 0 # 32 bit align gdt_desc.address boot_gdt_descr: .word __BOOT_DS+7 .long boot_gdt_table - __PAGE_OFFSET .word 0 # 32-bit align idt_desc.address idt_descr: .word IDT_ENTRIES*8-1 # idt contains 256 entries .long idt_table # boot GDT descriptor (later on used by CPU#0): .word 0 # 32 bit align gdt_desc.address cpu_gdt_descr: .word GDT_ENTRIES*8-1 .long cpu_gdt_table .fill NR_CPUS-1,8,0 # space for the other GDT descriptors ENTRY(__phys_offset) .long 0 /* * The boot_gdt_table must mirror the equivalent in setup.S and is * used only for booting. */ .align L1_CACHE_BYTES ENTRY(boot_gdt_table) .fill GDT_ENTRY_BOOT_CS,8,0 .quad 0x00cf9a000000ffff /* kernel 4GB code at 0x00000000 */ .quad 0x00cf92000000ffff /* kernel 4GB data at 0x00000000 */ /* * The Global Descriptor Table contains 28 quadwords, per-CPU. */ .align PAGE_SIZE_asm ENTRY(cpu_gdt_table) .quad 0x0000000000000000 /* NULL descriptor */ .quad 0x0000000000000000 /* 0x0b reserved */ .quad 0x0000000000000000 /* 0x13 reserved */ .quad 0x0000000000000000 /* 0x1b reserved */ .quad 0x0000000000000000 /* 0x20 unused */ .quad 0x0000000000000000 /* 0x28 unused */ .quad 0x0000000000000000 /* 0x33 TLS entry 1 */ .quad 0x0000000000000000 /* 0x3b TLS entry 2 */ .quad 0x0000000000000000 /* 0x43 TLS entry 3 */ .quad 0x0000000000000000 /* 0x4b reserved */ .quad 0x0000000000000000 /* 0x53 reserved */ .quad 0x0000000000000000 /* 0x5b reserved */ .quad 0x00cf9a000000ffff /* 0x60 kernel 4GB code at 0x00000000 */ .quad 0x00cf92000000ffff /* 0x68 kernel 4GB data at 0x00000000 */ .quad 0x00cffa000000ffff /* 0x73 user 4GB code at 0x00000000 */ .quad 0x00cff2000000ffff /* 0x7b user 4GB data at 0x00000000 */ .quad 0x0000000000000000 /* 0x80 TSS descriptor */ .quad 0x0000000000000000 /* 0x88 LDT descriptor */ /* Segments used for calling PnP BIOS */ .quad 0x00c09a0000000000 /* 0x90 32-bit code */ .quad 0x00809a0000000000 /* 0x98 16-bit code */ .quad 0x0080920000000000 /* 0xa0 16-bit data */ .quad 0x0080920000000000 /* 0xa8 16-bit data */ .quad 0x0080920000000000 /* 0xb0 16-bit data */ /* * The APM segments have byte granularity and their bases * and limits are set at run time. */ .quad 0x00409a0000000000 /* 0xb8 APM CS code */ .quad 0x00009a0000000000 /* 0xc0 APM CS 16 code (16 bit) */ .quad 0x0040920000000000 /* 0xc8 APM DS data */ .quad 0x0000000000000000 /* 0xd0 - unused */ .quad 0x0000000000000000 /* 0xd8 - unused */ .quad 0x0000000000000000 /* 0xe0 - unused */ .quad 0x0000000000000000 /* 0xe8 - unused */ .quad 0x0000000000000000 /* 0xf0 - unused */ .quad 0x0000000000000000 /* 0xf8 - GDT entry 31: double-fault TSS */ |
|
From: Itsuro O. <od...@us...> - 2005-12-15 04:53:58
|
Update of /cvsroot/mkdump/minikpatch/1.0/linus/2.6.9/arch/i386/mm In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv11351/arch/i386/mm Added Files: init.c pgtable.c Log Message: register mini-kernel v1 update1 to CVS --- NEW FILE: pgtable.c --- /* * linux/arch/i386/mm/pgtable.c */ #include <linux/config.h> #include <linux/sched.h> #include <linux/kernel.h> #include <linux/errno.h> #include <linux/mm.h> #include <linux/swap.h> #include <linux/smp.h> #include <linux/highmem.h> #include <linux/slab.h> #include <linux/pagemap.h> #include <linux/spinlock.h> #include <asm/system.h> #include <asm/pgtable.h> #include <asm/pgalloc.h> #include <asm/fixmap.h> #include <asm/e820.h> #include <asm/tlb.h> #include <asm/tlbflush.h> void show_mem(void) { int total = 0, reserved = 0; int shared = 0, cached = 0; int highmem = 0; struct page *page; pg_data_t *pgdat; unsigned long i; printk("Mem-info:\n"); show_free_areas(); printk("Free swap: %6ldkB\n", nr_swap_pages<<(PAGE_SHIFT-10)); for_each_pgdat(pgdat) { for (i = 0; i < pgdat->node_spanned_pages; ++i) { page = pgdat->node_mem_map + i; total++; if (PageHighMem(page)) highmem++; if (PageReserved(page)) reserved++; else if (PageSwapCache(page)) cached++; else if (page_count(page)) shared += page_count(page) - 1; } } printk("%d pages of RAM\n", total); printk("%d pages of HIGHMEM\n",highmem); printk("%d reserved pages\n",reserved); printk("%d pages shared\n",shared); printk("%d pages swap cached\n",cached); } /* * Associate a virtual page frame with a given physical page frame * and protection flags for that frame. */ static void set_pte_pfn(unsigned long vaddr, unsigned long pfn, pgprot_t flags) { pgd_t *pgd; pmd_t *pmd; pte_t *pte; pgd = swapper_pg_dir + pgd_index(vaddr); if (pgd_none(*pgd)) { BUG(); return; } pmd = pmd_offset(pgd, vaddr); if (pmd_none(*pmd)) { BUG(); return; } pte = pte_offset_kernel(pmd, vaddr); /* <pfn,flags> stored as-is, to permit clearing entries */ set_pte(pte, pfn_pte(pfn, flags)); /* * It's enough to flush this one mapping. * (PGE mappings get flushed as well) */ __flush_tlb_one(vaddr); } /* * Associate a large virtual page frame with a given physical page frame * and protection flags for that frame. pfn is for the base of the page, * vaddr is what the page gets mapped to - both must be properly aligned. * The pmd must already be instantiated. Assumes PAE mode. */ void set_pmd_pfn(unsigned long vaddr, unsigned long pfn, pgprot_t flags) { pgd_t *pgd; pmd_t *pmd; if (vaddr & (PMD_SIZE-1)) { /* vaddr is misaligned */ printk ("set_pmd_pfn: vaddr misaligned\n"); return; /* BUG(); */ } if (pfn & (PTRS_PER_PTE-1)) { /* pfn is misaligned */ printk ("set_pmd_pfn: pfn misaligned\n"); return; /* BUG(); */ } pgd = swapper_pg_dir + pgd_index(vaddr); if (pgd_none(*pgd)) { printk ("set_pmd_pfn: pgd_none\n"); return; /* BUG(); */ } pmd = pmd_offset(pgd, vaddr); set_pmd(pmd, pfn_pmd(pfn, flags)); /* * It's enough to flush this one mapping. * (PGE mappings get flushed as well) */ __flush_tlb_one(vaddr); } void __set_fixmap (enum fixed_addresses idx, unsigned long phys, pgprot_t flags) { unsigned long address = __fix_to_virt(idx); if (idx >= __end_of_fixed_addresses) { BUG(); return; } set_pte_pfn(address, phys >> PAGE_SHIFT, flags); } pte_t *pte_alloc_one_kernel(struct mm_struct *mm, unsigned long address) { pte_t *pte = (pte_t *)__get_free_page(GFP_KERNEL|__GFP_REPEAT); if (pte) clear_page(pte); return pte; } struct page *pte_alloc_one(struct mm_struct *mm, unsigned long address) { struct page *pte; #ifdef CONFIG_HIGHPTE pte = alloc_pages(GFP_KERNEL|__GFP_HIGHMEM|__GFP_REPEAT, 0); #else pte = alloc_pages(GFP_KERNEL|__GFP_REPEAT, 0); #endif if (pte) clear_highpage(pte); return pte; } void pmd_ctor(void *pmd, kmem_cache_t *cache, unsigned long flags) { memset(pmd, 0, PTRS_PER_PMD*sizeof(pmd_t)); } /* * List of all pgd's needed for non-PAE so it can invalidate entries * in both cached and uncached pgd's; not needed for PAE since the * kernel pmd is shared. If PAE were not to share the pmd a similar * tactic would be needed. This is essentially codepath-based locking * against pageattr.c; it is the unique case in which a valid change * of kernel pagetables can't be lazily synchronized by vmalloc faults. * vmalloc faults work because attached pagetables are never freed. * If the locking proves to be non-performant, a ticketing scheme with * checks at dup_mmap(), exec(), and other mmlist addition points * could be used. The locking scheme was chosen on the basis of * manfred's recommendations and having no core impact whatsoever. * -- wli */ spinlock_t pgd_lock = SPIN_LOCK_UNLOCKED; struct page *pgd_list; static inline void pgd_list_add(pgd_t *pgd) { struct page *page = virt_to_page(pgd); page->index = (unsigned long)pgd_list; if (pgd_list) pgd_list->private = (unsigned long)&page->index; pgd_list = page; page->private = (unsigned long)&pgd_list; } static inline void pgd_list_del(pgd_t *pgd) { struct page *next, **pprev, *page = virt_to_page(pgd); next = (struct page *)page->index; pprev = (struct page **)page->private; *pprev = next; if (next) next->private = (unsigned long)pprev; } void pgd_ctor(void *pgd, kmem_cache_t *cache, unsigned long unused) { unsigned long flags; if (PTRS_PER_PMD == 1) spin_lock_irqsave(&pgd_lock, flags); memcpy((pgd_t *)pgd + USER_PTRS_PER_PGD, swapper_pg_dir + USER_PTRS_PER_PGD, (PTRS_PER_PGD - USER_PTRS_PER_PGD) * sizeof(pgd_t)); if (PTRS_PER_PMD > 1) return; pgd_list_add(pgd); spin_unlock_irqrestore(&pgd_lock, flags); memset(pgd, 0, USER_PTRS_PER_PGD*sizeof(pgd_t)); } /* never called when PTRS_PER_PMD > 1 */ void pgd_dtor(void *pgd, kmem_cache_t *cache, unsigned long unused) { unsigned long flags; /* can be called from interrupt context */ spin_lock_irqsave(&pgd_lock, flags); pgd_list_del(pgd); spin_unlock_irqrestore(&pgd_lock, flags); } pgd_t *pgd_alloc(struct mm_struct *mm) { int i; pgd_t *pgd = kmem_cache_alloc(pgd_cache, GFP_KERNEL); if (PTRS_PER_PMD == 1 || !pgd) return pgd; for (i = 0; i < USER_PTRS_PER_PGD; ++i) { pmd_t *pmd = kmem_cache_alloc(pmd_cache, GFP_KERNEL); if (!pmd) goto out_oom; set_pgd(&pgd[i], __pgd(1 + __pa((u64)((u32)pmd)))); } return pgd; out_oom: for (i--; i >= 0; i--) kmem_cache_free(pmd_cache, (void *)__va(pgd_val(pgd[i])-1)); kmem_cache_free(pgd_cache, pgd); return NULL; } void pgd_free(pgd_t *pgd) { int i; /* in the PAE case user pgd entries are overwritten before usage */ if (PTRS_PER_PMD > 1) for (i = 0; i < USER_PTRS_PER_PGD; ++i) kmem_cache_free(pmd_cache, (void *)__va(pgd_val(pgd[i])-1)); /* in the non-PAE case, clear_page_tables() clears user pgd entries */ kmem_cache_free(pgd_cache, pgd); } /* * cheat kernel at page table entry layer */ extern unsigned long max_low_pfn; #ifdef CONFIG_X86_PAE void set_pte(pte_t *ptep, pte_t pte) { unsigned long long laddr = ((unsigned long long)pte.pte_high << 32) | pte.pte_low; if (pte.pte_low & _PAGE_PRESENT) { /* if not, swap entry */ if (laddr >> PAGE_SHIFT <= max_low_pfn) { /* note: max_low_pfn and true phys both <= ZONE_NORMAL */ pte.pte_low = true_phys(pte.pte_low); } } ptep->pte_high = pte.pte_high; smp_wmb(); /* actually not need */ ptep->pte_low = pte.pte_low; } void set_pte_atomic(pte_t *ptep, pte_t pte) { unsigned long long laddr = ((unsigned long long)pte.pte_high << 32) | pte.pte_low; if (pte.pte_low & _PAGE_PRESENT) { /* if not, swap entry */ if (laddr >> PAGE_SHIFT <= max_low_pfn) { /* note: max_low_pfn and true phys both <= ZONE_NORMAL */ pte.pte_low = true_phys(pte.pte_low); } } set_64bit((unsigned long long *)ptep, ((unsigned long long)pte.pte_high << 32) | pte.pte_low); } void set_pmd(pmd_t *pmdp, pmd_t pmd) { pmd.pmd = true_phys((unsigned long)pmd.pmd); set_64bit((unsigned long long *)pmdp, pmd.pmd); } void set_pgd(pgd_t *pgdp, pgd_t pgd) { pgd.pgd = true_phys((unsigned long)pgd.pgd); set_64bit((unsigned long long *)pgdp, pgd.pgd); } /* * XXX: ptep_get_and_clear: need change ?? */ unsigned long long pte_val(pte_t pte) { if (!pte.pte_high && (pte.pte_low & _PAGE_PRESENT)) { pte.pte_low = pseudo_phys(pte.pte_low); } return ((unsigned long long)pte.pte_high << 32) | pte.pte_low; } unsigned long long pmd_val(pmd_t pmd) { return pmd.pmd & _PAGE_PRESENT ? pseudo_phys((unsigned long)pmd.pmd) : pmd.pmd; } unsigned long long pgd_val(pgd_t pgd) { return pgd.pgd & _PAGE_PRESENT ? pseudo_phys((unsigned long)pgd.pgd) : pgd.pgd; } #else /* non PAE */ void set_pte(pte_t *ptep, pte_t pte) { ptep->pte_low = pte.pte_low & _PAGE_PRESENT ? true_phys(pte.pte_low) : pte.pte_low; } void set_pmd(pmd_t *pmdp, pmd_t pmd) { pmdp->pmd = pmd.pmd ? true_phys(pmd.pmd) : pmd.pmd; } void set_pgd(pgd_t *pgdp, pgd_t pgd) { pgdp->pgd = pgd.pgd ? true_phys(pgd.pgd) : pgd.pgd; } unsigned long pte_val(pte_t pte) { return pte.pte_low & _PAGE_PRESENT ? pseudo_phys(pte.pte_low) : pte.pte_low; } unsigned long pmd_val(pmd_t pmd) { return pmd.pmd & _PAGE_PRESENT ? pseudo_phys(pmd.pmd) : pmd.pmd; } unsigned long pgd_val(pgd_t pgd) { return pgd.pgd & _PAGE_PRESENT ? pseudo_phys(pgd.pgd) : pgd.pgd; } #endif /* CONFIG_X86_PAE */ unsigned long true_phys(unsigned long pa) { unsigned long *phys_table = (unsigned long *)(PAGE_OFFSET + MINIK_MEM_SEG_OFFSET); unsigned long pfn = pa >> PAGE_SHIFT; if (pfn >= max_low_pfn || (0xa0 <= pfn && pfn < 0x100)) { return pa; } else { return (pa & __PHYS_SEG_MASK) + phys_table[pa >> __PHYS_SEG_SHIFT]; } } unsigned long pseudo_phys(unsigned long pa) { unsigned long *phys_table = (unsigned long *)(PAGE_OFFSET + MINIK_MEM_SEG_OFFSET); int i; for (i = 0; i < MAX_MINIK_MEM_SEG; i++) { if (phys_table[i] == ((pa >> __PHYS_SEG_SHIFT) << __PHYS_SEG_SHIFT)) { return (pa & __PHYS_SEG_MASK) + ((unsigned long)i << __PHYS_SEG_SHIFT); } } return pa; } static pte_t *dump_pte = NULL; static unsigned long dump_vaddr; void dump_set_pte_pfn(unsigned long pfn) { if (!dump_pte) { pgd_t *pgd; pmd_t *pmd; dump_vaddr = __fix_to_virt(FIX_DUMP); pgd = swapper_pg_dir + pgd_index(dump_vaddr); if (pgd_none(*pgd)) { BUG(); return; } pmd = pmd_offset(pgd, dump_vaddr); if (pmd_none(*pmd)) { BUG(); return; } dump_pte = pte_offset_kernel(pmd, dump_vaddr); } /* set pfn asis. don't use set_pte */ *dump_pte = pfn_pte(pfn, PAGE_KERNEL); __flush_tlb_one(dump_vaddr); } --- NEW FILE: init.c --- /* * linux/arch/i386/mm/init.c * * Copyright (C) 1995 Linus Torvalds * * Support of BIGMEM added by Gerhard Wichert, Siemens AG, July 1999 */ #include <linux/config.h> #include <linux/module.h> #include <linux/signal.h> #include <linux/sched.h> #include <linux/kernel.h> #include <linux/errno.h> #include <linux/string.h> #include <linux/types.h> #include <linux/ptrace.h> #include <linux/mman.h> #include <linux/mm.h> #include <linux/hugetlb.h> #include <linux/swap.h> #include <linux/smp.h> #include <linux/init.h> #include <linux/highmem.h> #include <linux/pagemap.h> #include <linux/bootmem.h> #include <linux/slab.h> #include <linux/proc_fs.h> #include <linux/efi.h> #include <asm/processor.h> #include <asm/system.h> #include <asm/uaccess.h> #include <asm/pgtable.h> #include <asm/dma.h> #include <asm/fixmap.h> #include <asm/e820.h> #include <asm/apic.h> #include <asm/tlb.h> #include <asm/tlbflush.h> #include <asm/sections.h> unsigned int __VMALLOC_RESERVE = 128 << 20; DEFINE_PER_CPU(struct mmu_gather, mmu_gathers); unsigned long highstart_pfn, highend_pfn; static int noinline do_test_wp_bit(void); /* * Creates a middle page table and puts a pointer to it in the * given global directory entry. This only returns the gd entry * in non-PAE compilation mode, since the middle layer is folded. */ static pmd_t * __init one_md_table_init(pgd_t *pgd) { pmd_t *pmd_table; #ifdef CONFIG_X86_PAE pmd_table = (pmd_t *) alloc_bootmem_low_pages(PAGE_SIZE); set_pgd(pgd, __pgd(__pa(pmd_table) | _PAGE_PRESENT)); if (pmd_table != pmd_offset(pgd, 0)) BUG(); #else pmd_table = pmd_offset(pgd, 0); #endif return pmd_table; } /* * Create a page table and place a pointer to it in a middle page * directory entry. */ static pte_t * __init one_page_table_init(pmd_t *pmd) { if (pmd_none(*pmd)) { pte_t *page_table = (pte_t *) alloc_bootmem_low_pages(PAGE_SIZE); set_pmd(pmd, __pmd(__pa(page_table) | _PAGE_TABLE)); if (page_table != pte_offset_kernel(pmd, 0)) BUG(); return page_table; } return pte_offset_kernel(pmd, 0); } /* * This function initializes a certain range of kernel virtual memory * with new bootmem page tables, everywhere page tables are missing in * the given range. */ /* * NOTE: The pagetables are allocated contiguous on the physical space * so we can cache the place of the first one and move around without * checking the pgd every time. */ static void __init page_table_range_init (unsigned long start, unsigned long end, pgd_t *pgd_base) { pgd_t *pgd; pmd_t *pmd; int pgd_idx, pmd_idx; unsigned long vaddr; vaddr = start; pgd_idx = pgd_index(vaddr); pmd_idx = pmd_index(vaddr); pgd = pgd_base + pgd_idx; for ( ; (pgd_idx < PTRS_PER_PGD) && (vaddr != end); pgd++, pgd_idx++) { if (pgd_none(*pgd)) one_md_table_init(pgd); pmd = pmd_offset(pgd, vaddr); for (; (pmd_idx < PTRS_PER_PMD) && (vaddr != end); pmd++, pmd_idx++) { if (pmd_none(*pmd)) one_page_table_init(pmd); vaddr += PMD_SIZE; } pmd_idx = 0; } } static inline int is_kernel_text(unsigned long addr) { if (addr >= (unsigned long)_stext && addr <= (unsigned long)__init_end) return 1; return 0; } /* * This maps the physical memory to kernel virtual address space, a total * of max_low_pfn pages, by creating page tables starting from address * PAGE_OFFSET. */ static void __init kernel_physical_mapping_init(pgd_t *pgd_base) { unsigned long pfn; pgd_t *pgd; pmd_t *pmd; pte_t *pte; int pgd_idx, pmd_idx, pte_ofs; pgd_idx = pgd_index(PAGE_OFFSET); pgd = pgd_base + pgd_idx; pfn = 0; for (; pgd_idx < PTRS_PER_PGD; pgd++, pgd_idx++) { pmd = one_md_table_init(pgd); if (pfn >= max_low_pfn) continue; for (pmd_idx = 0; pmd_idx < PTRS_PER_PMD && pfn < max_low_pfn; pmd++, pmd_idx++) { unsigned int address = pfn * PAGE_SIZE + PAGE_OFFSET; pte = one_page_table_init(pmd); for (pte_ofs = 0; pte_ofs < PTRS_PER_PTE && pfn < max_low_pfn; pte++, pfn++, pte_ofs++) { if (is_kernel_text(address)) set_pte(pte, pfn_pte(pfn, PAGE_KERNEL_EXEC)); else set_pte(pte, pfn_pte(pfn, PAGE_KERNEL)); } } } } static inline int page_kills_ppro(unsigned long pagenr) { if (pagenr >= 0x70000 && pagenr <= 0x7003F) return 1; return 0; } extern int is_available_memory(efi_memory_desc_t *); static inline int page_is_ram(unsigned long pagenr) { int i; unsigned long addr, end; if (efi_enabled) { efi_memory_desc_t *md; for (i = 0; i < memmap.nr_map; i++) { md = &memmap.map[i]; if (!is_available_memory(md)) continue; addr = (md->phys_addr+PAGE_SIZE-1) >> PAGE_SHIFT; end = (md->phys_addr + (md->num_pages << EFI_PAGE_SHIFT)) >> PAGE_SHIFT; if ((pagenr >= addr) && (pagenr < end)) return 1; } return 0; } for (i = 0; i < e820.nr_map; i++) { if (e820.map[i].type != E820_RAM) /* not usable memory */ continue; /* * !!!FIXME!!! Some BIOSen report areas as RAM that * are not. Notably the 640->1Mb area. We need a sanity * check here. */ addr = (e820.map[i].addr+PAGE_SIZE-1) >> PAGE_SHIFT; end = (e820.map[i].addr+e820.map[i].size) >> PAGE_SHIFT; if ((pagenr >= addr) && (pagenr < end)) return 1; } return 0; } #ifdef CONFIG_HIGHMEM pte_t *kmap_pte; pgprot_t kmap_prot; EXPORT_SYMBOL(kmap_prot); EXPORT_SYMBOL(kmap_pte); #define kmap_get_fixmap_pte(vaddr) \ pte_offset_kernel(pmd_offset(pgd_offset_k(vaddr), (vaddr)), (vaddr)) void __init kmap_init(void) { unsigned long kmap_vstart; /* cache the first kmap pte */ kmap_vstart = __fix_to_virt(FIX_KMAP_BEGIN); kmap_pte = kmap_get_fixmap_pte(kmap_vstart); kmap_prot = PAGE_KERNEL; } void __init permanent_kmaps_init(pgd_t *pgd_base) { pgd_t *pgd; pmd_t *pmd; pte_t *pte; unsigned long vaddr; vaddr = PKMAP_BASE; page_table_range_init(vaddr, vaddr + PAGE_SIZE*LAST_PKMAP, pgd_base); pgd = swapper_pg_dir + pgd_index(vaddr); pmd = pmd_offset(pgd, vaddr); pte = pte_offset_kernel(pmd, vaddr); pkmap_page_table = pte; } void __init one_highpage_init(struct page *page, int pfn, int bad_ppro) { if (page_is_ram(pfn) && !(bad_ppro && page_kills_ppro(pfn))) { ClearPageReserved(page); set_bit(PG_highmem, &page->flags); set_page_count(page, 1); __free_page(page); totalhigh_pages++; } else SetPageReserved(page); } #ifndef CONFIG_DISCONTIGMEM void __init set_highmem_pages_init(int bad_ppro) { int pfn; for (pfn = highstart_pfn; pfn < highend_pfn; pfn++) one_highpage_init(pfn_to_page(pfn), pfn, bad_ppro); totalram_pages += totalhigh_pages; } #else extern void set_highmem_pages_init(int); #endif /* !CONFIG_DISCONTIGMEM */ #else #define kmap_init() do { } while (0) #define permanent_kmaps_init(pgd_base) do { } while (0) #define set_highmem_pages_init(bad_ppro) do { } while (0) #endif /* CONFIG_HIGHMEM */ unsigned long long __PAGE_KERNEL = _PAGE_KERNEL; unsigned long long __PAGE_KERNEL_EXEC = _PAGE_KERNEL_EXEC; #ifndef CONFIG_DISCONTIGMEM #define remap_numa_kva() do {} while (0) #else extern void __init remap_numa_kva(void); #endif static void __init pagetable_init (void) { unsigned long vaddr; pgd_t *pgd_base = swapper_pg_dir; #ifdef CONFIG_X86_PAE int i; /* Init entries of the first-level page table to the zero page */ for (i = 0; i < PTRS_PER_PGD; i++) set_pgd(pgd_base + i, __pgd(__pa(empty_zero_page) | _PAGE_PRESENT)); #endif /* Enable PSE if available */ if (cpu_has_pse) { set_in_cr4(X86_CR4_PSE); } /* Enable PGE if available */ if (cpu_has_pge) { set_in_cr4(X86_CR4_PGE); __PAGE_KERNEL |= _PAGE_GLOBAL; __PAGE_KERNEL_EXEC |= _PAGE_GLOBAL; } kernel_physical_mapping_init(pgd_base); remap_numa_kva(); /* * Fixed mappings, only the page table structure has to be * created - mappings will be set by set_fixmap(): */ vaddr = __fix_to_virt(__end_of_fixed_addresses - 1) & PMD_MASK; page_table_range_init(vaddr, 0, pgd_base); permanent_kmaps_init(pgd_base); #ifdef CONFIG_X86_PAE /* * Add low memory identity-mappings - SMP needs it when * starting up on an AP from real-mode. In the non-PAE * case we already have these mappings through head.S. * All user-space mappings are explicitly cleared after * SMP startup. */ pgd_base[0] = pgd_base[USER_PTRS_PER_PGD]; #endif } #if defined(CONFIG_PM_DISK) || defined(CONFIG_SOFTWARE_SUSPEND) /* * Swap suspend & friends need this for resume because things like the intel-agp * driver might have split up a kernel 4MB mapping. */ char __nosavedata swsusp_pg_dir[PAGE_SIZE] __attribute__ ((aligned (PAGE_SIZE))); static inline void save_pg_dir(void) { memcpy(swsusp_pg_dir, swapper_pg_dir, PAGE_SIZE); } #else static inline void save_pg_dir(void) { } #endif void zap_low_mappings (void) { int i; save_pg_dir(); /* * Zap initial low-memory mappings. * * Note that "pgd_clear()" doesn't do it for * us, because pgd_clear() is a no-op on i386. */ for (i = 0; i < USER_PTRS_PER_PGD; i++) #ifdef CONFIG_X86_PAE set_pgd(swapper_pg_dir+i, __pgd(1 + __pa(empty_zero_page))); #else set_pgd(swapper_pg_dir+i, __pgd(0)); #endif flush_tlb_all(); } #ifndef CONFIG_DISCONTIGMEM void __init zone_sizes_init(void) { unsigned long zones_size[MAX_NR_ZONES] = {0, 0, 0}; unsigned int max_dma, high, low; max_dma = virt_to_phys((char *)MAX_DMA_ADDRESS) >> PAGE_SHIFT; low = max_low_pfn; high = highend_pfn; if (low < max_dma) zones_size[ZONE_DMA] = low; else { zones_size[ZONE_DMA] = max_dma; zones_size[ZONE_NORMAL] = low - max_dma; #ifdef CONFIG_HIGHMEM zones_size[ZONE_HIGHMEM] = high - low; #endif } free_area_init(zones_size); } #else extern void zone_sizes_init(void); #endif /* !CONFIG_DISCONTIGMEM */ static int disable_nx __initdata = 1; u64 __supported_pte_mask = ~_PAGE_NX; /* * noexec = on|off * * Control non executable mappings. * * on Enable * off Disable */ static int __init noexec_setup(char *str) { if (!strncmp(str, "on",2) && cpu_has_nx) { __supported_pte_mask |= _PAGE_NX; disable_nx = 0; } else if (!strncmp(str,"off",3)) { disable_nx = 1; __supported_pte_mask &= ~_PAGE_NX; } return 1; } __setup("noexec=", noexec_setup); #ifdef CONFIG_X86_PAE int nx_enabled = 0; static void __init set_nx(void) { unsigned int v[4], l, h; if (cpu_has_pae && (cpuid_eax(0x80000000) > 0x80000001)) { cpuid(0x80000001, &v[0], &v[1], &v[2], &v[3]); if ((v[3] & (1 << 20)) && !disable_nx) { rdmsr(MSR_EFER, l, h); l |= EFER_NX; wrmsr(MSR_EFER, l, h); nx_enabled = 1; __supported_pte_mask |= _PAGE_NX; } } } /* * Enables/disables executability of a given kernel page and * returns the previous setting. */ int __init set_kernel_exec(unsigned long vaddr, int enable) { pte_t *pte; int ret = 1; if (!nx_enabled) goto out; pte = lookup_address(vaddr); BUG_ON(!pte); if (!pte_exec_kernel(*pte)) ret = 0; if (enable) pte->pte_high &= ~(1 << (_PAGE_BIT_NX - 32)); else pte->pte_high |= 1 << (_PAGE_BIT_NX - 32); __flush_tlb_all(); out: return ret; } #endif /* * paging_init() sets up the page tables - note that the first 8MB are * already mapped by head.S. * * This routines also unmaps the page at virtual kernel address 0, so * that we can trap those pesky NULL-reference errors in the kernel. */ void __init paging_init(void) { #ifdef CONFIG_X86_PAE set_nx(); if (nx_enabled) printk("NX (Execute Disable) protection: active\n"); #endif pagetable_init(); load_cr3(swapper_pg_dir); #ifdef CONFIG_X86_PAE /* * We will bail out later - printk doesn't work right now so * the user would just see a hanging kernel. */ if (cpu_has_pae) set_in_cr4(X86_CR4_PAE); #endif __flush_tlb_all(); kmap_init(); zone_sizes_init(); } /* * Test if the WP bit works in supervisor mode. It isn't supported on 386's * and also on some strange 486's (NexGen etc.). All 586+'s are OK. This * used to involve black magic jumps to work around some nasty CPU bugs, * but fortunately the switch to using exceptions got rid of all that. */ void __init test_wp_bit(void) { printk("Checking if this processor honours the WP bit even in supervisor mode... "); /* Any page-aligned address will do, the test is non-destructive */ __set_fixmap(FIX_WP_TEST, __pa(&swapper_pg_dir), PAGE_READONLY); boot_cpu_data.wp_works_ok = do_test_wp_bit(); clear_fixmap(FIX_WP_TEST); if (!boot_cpu_data.wp_works_ok) { printk("No.\n"); #ifdef CONFIG_X86_WP_WORKS_OK panic("This kernel doesn't support CPU's with broken WP. Recompile it for a 386!"); #endif } else { printk("Ok.\n"); } } #ifndef CONFIG_DISCONTIGMEM static void __init set_max_mapnr_init(void) { #ifdef CONFIG_HIGHMEM highmem_start_page = pfn_to_page(highstart_pfn); max_mapnr = num_physpages = highend_pfn; #else max_mapnr = num_physpages = max_low_pfn; #endif } #define __free_all_bootmem() free_all_bootmem() #else #define __free_all_bootmem() free_all_bootmem_node(NODE_DATA(0)) extern void set_max_mapnr_init(void); #endif /* !CONFIG_DISCONTIGMEM */ static struct kcore_list kcore_mem, kcore_vmalloc; void __init mem_init(void) { extern int ppro_with_ram_bug(void); int codesize, reservedpages, datasize, initsize; int tmp; int bad_ppro; #ifndef CONFIG_DISCONTIGMEM if (!mem_map) BUG(); #endif bad_ppro = ppro_with_ram_bug(); #ifdef CONFIG_HIGHMEM /* check that fixmap and pkmap do not overlap */ if (PKMAP_BASE+LAST_PKMAP*PAGE_SIZE >= FIXADDR_START) { printk(KERN_ERR "fixmap and kmap areas overlap - this will crash\n"); printk(KERN_ERR "pkstart: %lxh pkend: %lxh fixstart %lxh\n", PKMAP_BASE, PKMAP_BASE+LAST_PKMAP*PAGE_SIZE, FIXADDR_START); BUG(); } #endif set_max_mapnr_init(); #ifdef CONFIG_HIGHMEM high_memory = (void *) __va(highstart_pfn * PAGE_SIZE); #else high_memory = (void *) __va(max_low_pfn * PAGE_SIZE); #endif /* this will put all low memory onto the freelists */ totalram_pages += __free_all_bootmem(); reservedpages = 0; for (tmp = 0; tmp < max_low_pfn; tmp++) /* * Only count reserved RAM pages */ if (page_is_ram(tmp) && PageReserved(pfn_to_page(tmp))) reservedpages++; set_highmem_pages_init(bad_ppro); codesize = (unsigned long) &_etext - (unsigned long) &_text; datasize = (unsigned long) &_edata - (unsigned long) &_etext; initsize = (unsigned long) &__init_end - (unsigned long) &__init_begin; kclist_add(&kcore_mem, __va(0), max_low_pfn << PAGE_SHIFT); kclist_add(&kcore_vmalloc, (void *)VMALLOC_START, VMALLOC_END-VMALLOC_START); printk(KERN_INFO "Memory: %luk/%luk available (%dk kernel code, %dk reserved, %dk data, %dk init, %ldk highmem)\n", (unsigned long) nr_free_pages() << (PAGE_SHIFT-10), num_physpages << (PAGE_SHIFT-10), codesize >> 10, reservedpages << (PAGE_SHIFT-10), datasize >> 10, initsize >> 10, (unsigned long) (totalhigh_pages << (PAGE_SHIFT-10)) ); #ifdef CONFIG_X86_PAE if (!cpu_has_pae) panic("cannot execute a PAE-enabled kernel on a PAE-less CPU!"); #endif if (boot_cpu_data.wp_works_ok < 0) test_wp_bit(); /* * Subtle. SMP is doing it's boot stuff late (because it has to * fork idle threads) - but it also needs low mappings for the * protected-mode entry to work. We zap these entries only after * the WP-bit has been tested. */ #ifndef CONFIG_SMP zap_low_mappings(); #endif } kmem_cache_t *pgd_cache; kmem_cache_t *pmd_cache; void __init pgtable_cache_init(void) { if (PTRS_PER_PMD > 1) { pmd_cache = kmem_cache_create("pmd", PTRS_PER_PMD*sizeof(pmd_t), PTRS_PER_PMD*sizeof(pmd_t), 0, pmd_ctor, NULL); if (!pmd_cache) panic("pgtable_cache_init(): cannot create pmd cache"); } pgd_cache = kmem_cache_create("pgd", PTRS_PER_PGD*sizeof(pgd_t), PTRS_PER_PGD*sizeof(pgd_t), 0, pgd_ctor, PTRS_PER_PMD == 1 ? pgd_dtor : NULL); if (!pgd_cache) panic("pgtable_cache_init(): Cannot create pgd cache"); } /* * This function cannot be __init, since exceptions don't work in that * section. Put this after the callers, so that it cannot be inlined. */ static int noinline do_test_wp_bit(void) { char tmp_reg; int flag; __asm__ __volatile__( " movb %0,%1 \n" "1: movb %1,%0 \n" " xorl %2,%2 \n" "2: \n" ".section __ex_table,\"a\"\n" " .align 4 \n" " .long 1b,2b \n" ".previous \n" :"=m" (*(char *)fix_to_virt(FIX_WP_TEST)), "=q" (tmp_reg), "=r" (flag) :"2" (1) :"memory"); return flag; } void free_initmem(void) { unsigned long addr; addr = (unsigned long)(&__init_begin); for (; addr < (unsigned long)(&__init_end); addr += PAGE_SIZE) { ClearPageReserved(virt_to_page(addr)); set_page_count(virt_to_page(addr), 1); free_page(addr); totalram_pages++; } printk (KERN_INFO "Freeing unused kernel memory: %dk freed\n", (__init_end - __init_begin) >> 10); } #ifdef CONFIG_BLK_DEV_INITRD void free_initrd_mem(unsigned long start, unsigned long end) { if (start < end) printk (KERN_INFO "Freeing initrd memory: %ldk freed\n", (end - start) >> 10); for (; start < end; start += PAGE_SIZE) { ClearPageReserved(virt_to_page(start)); set_page_count(virt_to_page(start), 1); free_page(start); totalram_pages++; } } #endif |
|
From: Itsuro O. <od...@us...> - 2005-12-15 04:53:58
|
Update of /cvsroot/mkdump/minikpatch/1.0/linus/2.6.9/include/asm-i386 In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv11351/include/asm-i386 Added Files: dma-mapping.h fixmap.h page.h pgtable-2level.h pgtable-3level.h processor.h Log Message: register mini-kernel v1 update1 to CVS --- NEW FILE: page.h --- #ifndef _I386_PAGE_H #define _I386_PAGE_H /* PAGE_SHIFT determines the page size */ #define PAGE_SHIFT 12 #define PAGE_SIZE (1UL << PAGE_SHIFT) #define PAGE_MASK (~(PAGE_SIZE-1)) #define LARGE_PAGE_MASK (~(LARGE_PAGE_SIZE-1)) #define LARGE_PAGE_SIZE (1UL << PMD_SHIFT) #ifdef __KERNEL__ #ifndef __ASSEMBLY__ #include <linux/config.h> #ifdef CONFIG_X86_USE_3DNOW #include <asm/mmx.h> #define clear_page(page) mmx_clear_page((void *)(page)) #define copy_page(to,from) mmx_copy_page(to,from) #else /* * On older X86 processors it's not a win to use MMX here it seems. * Maybe the K6-III ? */ #define clear_page(page) memset((void *)(page), 0, PAGE_SIZE) #define copy_page(to,from) memcpy((void *)(to), (void *)(from), PAGE_SIZE) #endif #define clear_user_page(page, vaddr, pg) clear_page(page) #define copy_user_page(to, from, vaddr, pg) copy_page(to, from) /* * These are used to make use of C type-checking.. */ #ifdef CONFIG_X86_PAE extern unsigned long long __supported_pte_mask; extern int nx_enabled; typedef struct { unsigned long pte_low, pte_high; } pte_t; typedef struct { unsigned long long pmd; } pmd_t; typedef struct { unsigned long long pgd; } pgd_t; typedef struct { unsigned long long pgprot; } pgprot_t; #ifdef CONFIG_DUMP_MINI_KERNEL extern unsigned long long pte_val(pte_t); #else #define pte_val(x) ((x).pte_low | ((unsigned long long)(x).pte_high << 32)) #endif #define HPAGE_SHIFT 21 #else #define nx_enabled 0 typedef struct { unsigned long pte_low; } pte_t; typedef struct { unsigned long pmd; } pmd_t; typedef struct { unsigned long pgd; } pgd_t; typedef struct { unsigned long pgprot; } pgprot_t; #define boot_pte_t pte_t /* or would you rather have a typedef */ #ifdef CONFIG_DUMP_MINI_KERNEL extern unsigned long pte_val(pte_t); #else #define pte_val(x) ((x).pte_low) #endif #define HPAGE_SHIFT 22 #endif #define PTE_MASK PAGE_MASK #ifdef CONFIG_HUGETLB_PAGE #define HPAGE_SIZE ((1UL) << HPAGE_SHIFT) #define HPAGE_MASK (~(HPAGE_SIZE - 1)) #define HUGETLB_PAGE_ORDER (HPAGE_SHIFT - PAGE_SHIFT) #endif #ifdef CONFIG_DUMP_MINI_KERNEL #ifdef CONFIG_X86_PAE extern unsigned long long pmd_val(pmd_t); extern unsigned long long pgd_val(pgd_t); #else extern unsigned long pmd_val(pmd_t); extern unsigned long pgd_val(pgd_t); #endif #else #define pmd_val(x) ((x).pmd) #define pgd_val(x) ((x).pgd) #endif #define pgprot_val(x) ((x).pgprot) #define __pte(x) ((pte_t) { (x) } ) #define __pmd(x) ((pmd_t) { (x) } ) #define __pgd(x) ((pgd_t) { (x) } ) #define __pgprot(x) ((pgprot_t) { (x) } ) #endif /* !__ASSEMBLY__ */ /* to align the pointer to the (next) page boundary */ #define PAGE_ALIGN(addr) (((addr)+PAGE_SIZE-1)&PAGE_MASK) /* * This handles the memory map.. We could make this a config * option, but too many people screw it up, and too few need * it. * * A __PAGE_OFFSET of 0xC0000000 means that the kernel has * a virtual address space of one gigabyte, which limits the * amount of physical memory you can use to about 950MB. * * If you want more physical memory than this then see the CONFIG_HIGHMEM4G * and CONFIG_HIGHMEM64G options in the kernel configuration. */ #ifndef __ASSEMBLY__ /* * This much address space is reserved for vmalloc() and iomap() * as well as fixmap mappings. */ extern unsigned int __VMALLOC_RESERVE; /* Pure 2^n version of get_order */ static __inline__ int get_order(unsigned long size) { int order; size = (size-1) >> (PAGE_SHIFT-1); order = -1; do { size >>= 1; order++; } while (size); return order; } extern int sysctl_legacy_va_layout; #endif /* __ASSEMBLY__ */ #ifdef __ASSEMBLY__ #define __PAGE_OFFSET (0xC0000000) #else #define __PAGE_OFFSET (0xC0000000UL) #endif #define PAGE_OFFSET ((unsigned long)__PAGE_OFFSET) #define VMALLOC_RESERVE ((unsigned long)__VMALLOC_RESERVE) #define MAXMEM (-__PAGE_OFFSET-__VMALLOC_RESERVE) #define __pa(x) ((unsigned long)(x)-PAGE_OFFSET) #define __va(x) ((void *)((unsigned long)(x)+PAGE_OFFSET)) #define pfn_to_kaddr(pfn) __va((pfn) << PAGE_SHIFT) #ifndef CONFIG_DISCONTIGMEM #define pfn_to_page(pfn) (mem_map + (pfn)) #define page_to_pfn(page) ((unsigned long)((page) - mem_map)) #define pfn_valid(pfn) ((pfn) < max_mapnr) #endif /* !CONFIG_DISCONTIGMEM */ #define virt_to_page(kaddr) pfn_to_page(__pa(kaddr) >> PAGE_SHIFT) #define virt_addr_valid(kaddr) pfn_valid(__pa(kaddr) >> PAGE_SHIFT) #define VM_DATA_DEFAULT_FLAGS \ (VM_READ | VM_WRITE | \ ((current->personality & READ_IMPLIES_EXEC) ? VM_EXEC : 0 ) | \ VM_MAYREAD | VM_MAYWRITE | VM_MAYEXEC) #ifndef __ASSEMBLY__ #ifdef CONFIG_DUMP_MINI_KERNEL #define MINIK_MEM_SEG_OFFSET 4096 #define MINIK_DUMP_HEADER_OFFSET 8192 #define MEM_SEG_OFFSET 12288 #define MAX_MINIK_MEM_SEG 32 /* offset of TRUE physical addresses. defined in head.S */ extern unsigned long __phys_offset; #define __PHYS_SEG_SHIFT 22 #define __PHYS_SEG_MASK (0x3fffffUL) extern unsigned long true_phys(unsigned long pa); extern unsigned long pseudo_phys(unsigned long pa); #else /* DUMP_MINI_KERNEL */ #define true_phys(x) (x) #define pseudo_phys(x) (x) #endif /* DUMP_MINI_KERNEL */ #endif /* __ASSEMBLY */ #endif /* __KERNEL__ */ #endif /* _I386_PAGE_H */ --- NEW FILE: dma-mapping.h --- #ifndef _ASM_I386_DMA_MAPPING_H #define _ASM_I386_DMA_MAPPING_H #include <linux/mm.h> #include <asm/cache.h> #include <asm/io.h> #include <asm/scatterlist.h> #define dma_alloc_noncoherent(d, s, h, f) dma_alloc_coherent(d, s, h, f) #define dma_free_noncoherent(d, s, v, h) dma_free_coherent(d, s, v, h) void *dma_alloc_coherent(struct device *dev, size_t size, dma_addr_t *dma_handle, int flag); void dma_free_coherent(struct device *dev, size_t size, void *vaddr, dma_addr_t dma_handle); static inline dma_addr_t dma_map_single(struct device *dev, void *ptr, size_t size, enum dma_data_direction direction) { BUG_ON(direction == DMA_NONE); flush_write_buffers(); return true_phys(virt_to_phys(ptr)); } static inline void dma_unmap_single(struct device *dev, dma_addr_t dma_addr, size_t size, enum dma_data_direction direction) { BUG_ON(direction == DMA_NONE); } static inline int dma_map_sg(struct device *dev, struct scatterlist *sg, int nents, enum dma_data_direction direction) { int i; BUG_ON(direction == DMA_NONE); for (i = 0; i < nents; i++ ) { BUG_ON(!sg[i].page); sg[i].dma_address = true_phys(page_to_phys(sg[i].page)) + sg[i].offset; } flush_write_buffers(); return nents; } static inline dma_addr_t dma_map_page(struct device *dev, struct page *page, unsigned long offset, size_t size, enum dma_data_direction direction) { BUG_ON(direction == DMA_NONE); return true_phys(page_to_phys(page)) + offset; } static inline void dma_unmap_page(struct device *dev, dma_addr_t dma_address, size_t size, enum dma_data_direction direction) { BUG_ON(direction == DMA_NONE); } static inline void dma_unmap_sg(struct device *dev, struct scatterlist *sg, int nhwentries, enum dma_data_direction direction) { BUG_ON(direction == DMA_NONE); } static inline void dma_sync_single_for_cpu(struct device *dev, dma_addr_t dma_handle, size_t size, enum dma_data_direction direction) { } static inline void dma_sync_single_for_device(struct device *dev, dma_addr_t dma_handle, size_t size, enum dma_data_direction direction) { flush_write_buffers(); } static inline void dma_sync_single_range_for_cpu(struct device *dev, dma_addr_t dma_handle, unsigned long offset, size_t size, enum dma_data_direction direction) { } static inline void dma_sync_single_range_for_device(struct device *dev, dma_addr_t dma_handle, unsigned long offset, size_t size, enum dma_data_direction direction) { flush_write_buffers(); } static inline void dma_sync_sg_for_cpu(struct device *dev, struct scatterlist *sg, int nelems, enum dma_data_direction direction) { } static inline void dma_sync_sg_for_device(struct device *dev, struct scatterlist *sg, int nelems, enum dma_data_direction direction) { flush_write_buffers(); } static inline int dma_mapping_error(dma_addr_t dma_addr) { return 0; } static inline int dma_supported(struct device *dev, u64 mask) { /* * we fall back to GFP_DMA when the mask isn't all 1s, * so we can't guarantee allocations that must be * within a tighter range than GFP_DMA.. */ if(mask < 0x00ffffff) return 0; return 1; } static inline int dma_set_mask(struct device *dev, u64 mask) { if(!dev->dma_mask || !dma_supported(dev, mask)) return -EIO; *dev->dma_mask = mask; return 0; } static inline int dma_get_cache_alignment(void) { /* no easy way to get cache size on all x86, so return the * maximum possible, to be safe */ return (1 << L1_CACHE_SHIFT_MAX); } #define dma_is_consistent(d) (1) static inline void dma_cache_sync(void *vaddr, size_t size, enum dma_data_direction direction) { flush_write_buffers(); } #define ARCH_HAS_DMA_DECLARE_COHERENT_MEMORY extern int dma_declare_coherent_memory(struct device *dev, dma_addr_t bus_addr, dma_addr_t device_addr, size_t size, int flags); extern void dma_release_declared_memory(struct device *dev); extern void * dma_mark_declared_memory_occupied(struct device *dev, dma_addr_t device_addr, size_t size); #endif --- NEW FILE: pgtable-3level.h --- #ifndef _I386_PGTABLE_3LEVEL_H #define _I386_PGTABLE_3LEVEL_H /* * Intel Physical Address Extension (PAE) Mode - three-level page * tables on PPro+ CPUs. * * Copyright (C) 1999 Ingo Molnar <mi...@re...> */ #define pte_ERROR(e) \ printk("%s:%d: bad pte %p(%08lx%08lx).\n", __FILE__, __LINE__, &(e), (e).pte_high, (e).pte_low) #define pmd_ERROR(e) \ printk("%s:%d: bad pmd %p(%016Lx).\n", __FILE__, __LINE__, &(e), pmd_val(e)) #define pgd_ERROR(e) \ printk("%s:%d: bad pgd %p(%016Lx).\n", __FILE__, __LINE__, &(e), pgd_val(e)) static inline int pgd_none(pgd_t pgd) { return 0; } static inline int pgd_bad(pgd_t pgd) { return 0; } static inline int pgd_present(pgd_t pgd) { return 1; } /* * Is the pte executable? */ static inline int pte_x(pte_t pte) { return !(pte_val(pte) & _PAGE_NX); } /* * All present user-pages with !NX bit are user-executable: */ static inline int pte_exec(pte_t pte) { return pte_user(pte) && pte_x(pte); } /* * All present pages with !NX bit are kernel-executable: */ static inline int pte_exec_kernel(pte_t pte) { return pte_x(pte); } /* Rules for using set_pte: the pte being assigned *must* be * either not present or in a state where the hardware will * not attempt to update the pte. In places where this is * not possible, use pte_get_and_clear to obtain the old pte * value and then use set_pte to update it. -ben */ #ifdef CONFIG_DUMP_MINI_KERNEL extern void set_pte(pte_t *, pte_t); extern void set_pte_atomic(pte_t *, pte_t); extern void set_pmd(pmd_t *, pmd_t); extern void set_pgd(pgd_t *, pgd_t); #else static inline void set_pte(pte_t *ptep, pte_t pte) { ptep->pte_high = pte.pte_high; smp_wmb(); ptep->pte_low = pte.pte_low; } #define __HAVE_ARCH_SET_PTE_ATOMIC #define set_pte_atomic(pteptr,pteval) \ set_64bit((unsigned long long *)(pteptr),pte_val(pteval)) #define set_pmd(pmdptr,pmdval) \ set_64bit((unsigned long long *)(pmdptr),pmd_val(pmdval)) #define set_pgd(pgdptr,pgdval) \ set_64bit((unsigned long long *)(pgdptr),pgd_val(pgdval)) #endif /* * Pentium-II erratum A13: in PAE mode we explicitly have to flush * the TLB via cr3 if the top-level pgd is changed... * We do not let the generic code free and clear pgd entries due to * this erratum. */ static inline void pgd_clear (pgd_t * pgd) { } #define pgd_page(pgd) \ ((unsigned long) __va(pgd_val(pgd) & PAGE_MASK)) /* Find an entry in the second-level page table.. */ #define pmd_offset(dir, address) ((pmd_t *) pgd_page(*(dir)) + \ pmd_index(address)) static inline pte_t ptep_get_and_clear(pte_t *ptep) { pte_t res; /* xchg acts as a barrier before the setting of the high bits */ res.pte_low = xchg(&ptep->pte_low, 0); res.pte_high = ptep->pte_high; ptep->pte_high = 0; return res; } static inline int pte_same(pte_t a, pte_t b) { return a.pte_low == b.pte_low && a.pte_high == b.pte_high; } #define pte_page(x) pfn_to_page(pte_pfn(x)) static inline int pte_none(pte_t pte) { return !pte.pte_low && !pte.pte_high; } static inline unsigned long pte_pfn(pte_t pte) { return (pte.pte_low >> PAGE_SHIFT) | (pte.pte_high << (32 - PAGE_SHIFT)); } extern unsigned long long __supported_pte_mask; static inline pte_t pfn_pte(unsigned long page_nr, pgprot_t pgprot) { pte_t pte; pte.pte_high = (page_nr >> (32 - PAGE_SHIFT)) | \ (pgprot_val(pgprot) >> 32); pte.pte_high &= (__supported_pte_mask >> 32); pte.pte_low = ((page_nr << PAGE_SHIFT) | pgprot_val(pgprot)) & \ __supported_pte_mask; return pte; } static inline pmd_t pfn_pmd(unsigned long page_nr, pgprot_t pgprot) { return __pmd((((unsigned long long)page_nr << PAGE_SHIFT) | \ pgprot_val(pgprot)) & __supported_pte_mask); } /* * Bits 0, 6 and 7 are taken in the low part of the pte, * put the 32 bits of offset into the high part. */ #define pte_to_pgoff(pte) ((pte).pte_high) #define pgoff_to_pte(off) ((pte_t) { _PAGE_FILE, (off) }) #define PTE_FILE_MAX_BITS 32 /* Encode and de-code a swap entry */ #define __swp_type(x) (((x).val) & 0x1f) #define __swp_offset(x) ((x).val >> 5) #define __swp_entry(type, offset) ((swp_entry_t){(type) | (offset) << 5}) #define __pte_to_swp_entry(pte) ((swp_entry_t){ (pte).pte_high }) #define __swp_entry_to_pte(x) ((pte_t){ 0, (x).val }) #endif /* _I386_PGTABLE_3LEVEL_H */ --- NEW FILE: pgtable-2level.h --- #ifndef _I386_PGTABLE_2LEVEL_H #define _I386_PGTABLE_2LEVEL_H #define pte_ERROR(e) \ printk("%s:%d: bad pte %08lx.\n", __FILE__, __LINE__, (e).pte_low) #define pmd_ERROR(e) \ printk("%s:%d: bad pmd %08lx.\n", __FILE__, __LINE__, pmd_val(e)) #define pgd_ERROR(e) \ printk("%s:%d: bad pgd %08lx.\n", __FILE__, __LINE__, pgd_val(e)) /* * The "pgd_xxx()" functions here are trivial for a folded two-level * setup: the pgd is never bad, and a pmd always exists (as it's folded * into the pgd entry) */ static inline int pgd_none(pgd_t pgd) { return 0; } static inline int pgd_bad(pgd_t pgd) { return 0; } static inline int pgd_present(pgd_t pgd) { return 1; } #define pgd_clear(xp) do { } while (0) /* * Certain architectures need to do special things when PTEs * within a page table are directly modified. Thus, the following * hook is made available. */ #ifdef CONFIG_DUMP_MINI_KERNEL extern void set_pte(pte_t *, pte_t); #else #define set_pte(pteptr, pteval) (*(pteptr) = pteval) #endif #define set_pte_atomic(pteptr, pteval) set_pte(pteptr,pteval) /* * (pmds are folded into pgds so this doesn't get actually called, * but the define is needed for a generic inline function.) */ #ifdef CONFIG_DUMP_MINI_KERNEL extern void set_pmd(pmd_t *, pmd_t); extern void set_pgd(pgd_t *, pgd_t); #else #define set_pmd(pmdptr, pmdval) (*(pmdptr) = pmdval) #define set_pgd(pgdptr, pgdval) (*(pgdptr) = pgdval) #endif #define pgd_page(pgd) \ ((unsigned long) __va(pgd_val(pgd) & PAGE_MASK)) static inline pmd_t * pmd_offset(pgd_t * dir, unsigned long address) { return (pmd_t *) dir; } #define ptep_get_and_clear(xp) __pte(xchg(&(xp)->pte_low, 0)) #define pte_same(a, b) ((a).pte_low == (b).pte_low) #define pte_page(x) pfn_to_page(pte_pfn(x)) #define pte_none(x) (!(x).pte_low) #define pte_pfn(x) ((unsigned long)(((x).pte_low >> PAGE_SHIFT))) #define pfn_pte(pfn, prot) __pte(((pfn) << PAGE_SHIFT) | pgprot_val(prot)) #define pfn_pmd(pfn, prot) __pmd(((pfn) << PAGE_SHIFT) | pgprot_val(prot)) /* * All present user pages are user-executable: */ static inline int pte_exec(pte_t pte) { return pte_user(pte); } /* * All present pages are kernel-executable: */ static inline int pte_exec_kernel(pte_t pte) { return 1; } /* * Bits 0, 6 and 7 are taken, split up the 29 bits of offset * into this range: */ #define PTE_FILE_MAX_BITS 29 #define pte_to_pgoff(pte) \ ((((pte).pte_low >> 1) & 0x1f ) + (((pte).pte_low >> 8) << 5 )) #define pgoff_to_pte(off) \ ((pte_t) { (((off) & 0x1f) << 1) + (((off) >> 5) << 8) + _PAGE_FILE }) /* Encode and de-code a swap entry */ #define __swp_type(x) (((x).val >> 1) & 0x1f) #define __swp_offset(x) ((x).val >> 8) #define __swp_entry(type, offset) ((swp_entry_t) { ((type) << 1) | ((offset) << 8) }) #define __pte_to_swp_entry(pte) ((swp_entry_t) { (pte).pte_low }) #define __swp_entry_to_pte(x) ((pte_t) { (x).val }) #endif /* _I386_PGTABLE_2LEVEL_H */ --- NEW FILE: fixmap.h --- /* * fixmap.h: compile-time virtual memory allocation * * This file is subject to the terms and conditions of the GNU General Public * License. See the file "COPYING" in the main directory of this archive * for more details. * * Copyright (C) 1998 Ingo Molnar * * Support of BIGMEM added by Gerhard Wichert, Siemens AG, July 1999 */ #ifndef _ASM_FIXMAP_H #define _ASM_FIXMAP_H #include <linux/config.h> /* used by vmalloc.c, vsyscall.lds.S. * * Leave one empty page between vmalloc'ed areas and * the start of the fixmap. */ #define __FIXADDR_TOP 0xfffff000 #ifndef __ASSEMBLY__ #include <linux/kernel.h> #include <asm/acpi.h> #include <asm/apicdef.h> #include <asm/page.h> #ifdef CONFIG_HIGHMEM #include <linux/threads.h> #include <asm/kmap_types.h> #endif /* * Here we define all the compile-time 'special' virtual * addresses. The point is to have a constant address at * compile time, but to set the physical address only * in the boot process. We allocate these special addresses * from the end of virtual memory (0xfffff000) backwards. * Also this lets us do fail-safe vmalloc(), we * can guarantee that these special addresses and * vmalloc()-ed addresses never overlap. * * these 'compile-time allocated' memory buffers are * fixed-size 4k pages. (or larger if used with an increment * highger than 1) use fixmap_set(idx,phys) to associate * physical memory with fixmap indices. * * TLB entries of such buffers will not be flushed across * task switches. */ enum fixed_addresses { FIX_HOLE, FIX_VSYSCALL, #ifdef CONFIG_X86_LOCAL_APIC FIX_APIC_BASE, /* local (CPU) APIC) -- required for SMP or not */ #endif #ifdef CONFIG_X86_IO_APIC FIX_IO_APIC_BASE_0, FIX_IO_APIC_BASE_END = FIX_IO_APIC_BASE_0 + MAX_IO_APICS-1, #endif #ifdef CONFIG_X86_VISWS_APIC FIX_CO_CPU, /* Cobalt timer */ FIX_CO_APIC, /* Cobalt APIC Redirection Table */ FIX_LI_PCIA, /* Lithium PCI Bridge A */ FIX_LI_PCIB, /* Lithium PCI Bridge B */ #endif #ifdef CONFIG_X86_F00F_BUG FIX_F00F_IDT, /* Virtual mapping for IDT */ #endif #ifdef CONFIG_X86_CYCLONE_TIMER FIX_CYCLONE_TIMER, /*cyclone timer register*/ #endif #ifdef CONFIG_HIGHMEM FIX_KMAP_BEGIN, /* reserved pte's for temporary kernel mappings */ FIX_KMAP_END = FIX_KMAP_BEGIN+(KM_TYPE_NR*NR_CPUS)-1, #endif #ifdef CONFIG_ACPI_BOOT FIX_ACPI_BEGIN, FIX_ACPI_END = FIX_ACPI_BEGIN + FIX_ACPI_PAGES - 1, #endif #ifdef CONFIG_PCI_MMCONFIG FIX_PCIE_MCFG, #endif __end_of_permanent_fixed_addresses, /* temporary boot-time mappings, used before ioremap() is functional */ #define NR_FIX_BTMAPS 16 FIX_BTMAP_END = __end_of_permanent_fixed_addresses, FIX_BTMAP_BEGIN = FIX_BTMAP_END + NR_FIX_BTMAPS - 1, FIX_WP_TEST, FIX_DUMP, __end_of_fixed_addresses }; extern void __set_fixmap (enum fixed_addresses idx, unsigned long phys, pgprot_t flags); #define set_fixmap(idx, phys) \ __set_fixmap(idx, phys, PAGE_KERNEL) /* * Some hardware wants to get fixmapped without caching. */ #define set_fixmap_nocache(idx, phys) \ __set_fixmap(idx, phys, PAGE_KERNEL_NOCACHE) #define clear_fixmap(idx) \ __set_fixmap(idx, 0, __pgprot(0)) #define FIXADDR_TOP ((unsigned long)__FIXADDR_TOP) #define __FIXADDR_SIZE (__end_of_permanent_fixed_addresses << PAGE_SHIFT) #define FIXADDR_START (FIXADDR_TOP - __FIXADDR_SIZE) #define __fix_to_virt(x) (FIXADDR_TOP - ((x) << PAGE_SHIFT)) #define __virt_to_fix(x) ((FIXADDR_TOP - ((x)&PAGE_MASK)) >> PAGE_SHIFT) /* * This is the range that is readable by user mode, and things * acting like user mode such as get_user_pages. */ #define FIXADDR_USER_START (__fix_to_virt(FIX_VSYSCALL)) #define FIXADDR_USER_END (FIXADDR_USER_START + PAGE_SIZE) extern void __this_fixmap_does_not_exist(void); /* * 'index to address' translation. If anyone tries to use the idx * directly without tranlation, we catch the bug with a NULL-deference * kernel oops. Illegal ranges of incoming indices are caught too. */ static __always_inline unsigned long fix_to_virt(const unsigned int idx) { /* * this branch gets completely eliminated after inlining, * except when someone tries to use fixaddr indices in an * illegal way. (such as mixing up address types or using * out-of-range indices). * * If it doesn't get removed, the linker will complain * loudly with a reasonably clear error message.. */ if (idx >= __end_of_fixed_addresses) __this_fixmap_does_not_exist(); return __fix_to_virt(idx); } static inline unsigned long virt_to_fix(const unsigned long vaddr) { BUG_ON(vaddr >= FIXADDR_TOP || vaddr < FIXADDR_START); return __virt_to_fix(vaddr); } #endif /* !__ASSEMBLY__ */ #endif --- NEW FILE: processor.h --- /* * include/asm-i386/processor.h * * Copyright (C) 1994 Linus Torvalds */ #ifndef __ASM_I386_PROCESSOR_H #define __ASM_I386_PROCESSOR_H #include <asm/vm86.h> #include <asm/math_emu.h> #include <asm/segment.h> #include <asm/page.h> #include <asm/types.h> #include <asm/sigcontext.h> #include <asm/cpufeature.h> #include <asm/msr.h> #include <asm/system.h> #include <linux/cache.h> #include <linux/config.h> #include <linux/threads.h> #include <asm/percpu.h> /* flag for disabling the tsc */ extern int tsc_disable; struct desc_struct { unsigned long a,b; }; #define desc_empty(desc) \ (!((desc)->a + (desc)->b)) #define desc_equal(desc1, desc2) \ (((desc1)->a == (desc2)->a) && ((desc1)->b == (desc2)->b)) /* * Default implementation of macro that returns current * instruction pointer ("program counter"). */ #define current_text_addr() ({ void *pc; __asm__("movl $1f,%0\n1:":"=g" (pc)); pc; }) /* * CPU type and hardware bug flags. Kept separately for each CPU. * Members of this structure are referenced in head.S, so think twice * before touching them. [mj] */ struct cpuinfo_x86 { __u8 x86; /* CPU family */ __u8 x86_vendor; /* CPU vendor */ __u8 x86_model; __u8 x86_mask; char wp_works_ok; /* It doesn't on 386's */ char hlt_works_ok; /* Problems on some 486Dx4's and old 386's */ char hard_math; char rfu; int cpuid_level; /* Maximum supported CPUID level, -1=no CPUID */ unsigned long x86_capability[NCAPINTS]; char x86_vendor_id[16]; char x86_model_id[64]; int x86_cache_size; /* in KB - valid for CPUS which support this call */ int x86_cache_alignment; /* In bytes */ int fdiv_bug; int f00f_bug; int coma_bug; unsigned long loops_per_jiffy; } __attribute__((__aligned__(SMP_CACHE_BYTES))); #define X86_VENDOR_INTEL 0 #define X86_VENDOR_CYRIX 1 #define X86_VENDOR_AMD 2 #define X86_VENDOR_UMC 3 #define X86_VENDOR_NEXGEN 4 #define X86_VENDOR_CENTAUR 5 #define X86_VENDOR_RISE 6 #define X86_VENDOR_TRANSMETA 7 #define X86_VENDOR_NSC 8 #define X86_VENDOR_NUM 9 #define X86_VENDOR_UNKNOWN 0xff /* * capabilities of CPUs */ extern struct cpuinfo_x86 boot_cpu_data; extern struct cpuinfo_x86 new_cpu_data; extern struct tss_struct doublefault_tss; DECLARE_PER_CPU(struct tss_struct, init_tss); #ifdef CONFIG_SMP extern struct cpuinfo_x86 cpu_data[]; #define current_cpu_data cpu_data[smp_processor_id()] #else #define cpu_data (&boot_cpu_data) #define current_cpu_data boot_cpu_data #endif extern char ignore_fpu_irq; extern void identify_cpu(struct cpuinfo_x86 *); extern void print_cpu_info(struct cpuinfo_x86 *); extern void dodgy_tsc(void); /* * EFLAGS bits */ #define X86_EFLAGS_CF 0x00000001 /* Carry Flag */ #define X86_EFLAGS_PF 0x00000004 /* Parity Flag */ #define X86_EFLAGS_AF 0x00000010 /* Auxillary carry Flag */ #define X86_EFLAGS_ZF 0x00000040 /* Zero Flag */ #define X86_EFLAGS_SF 0x00000080 /* Sign Flag */ #define X86_EFLAGS_TF 0x00000100 /* Trap Flag */ #define X86_EFLAGS_IF 0x00000200 /* Interrupt Flag */ #define X86_EFLAGS_DF 0x00000400 /* Direction Flag */ #define X86_EFLAGS_OF 0x00000800 /* Overflow Flag */ #define X86_EFLAGS_IOPL 0x00003000 /* IOPL mask */ #define X86_EFLAGS_NT 0x00004000 /* Nested Task */ #define X86_EFLAGS_RF 0x00010000 /* Resume Flag */ #define X86_EFLAGS_VM 0x00020000 /* Virtual Mode */ #define X86_EFLAGS_AC 0x00040000 /* Alignment Check */ #define X86_EFLAGS_VIF 0x00080000 /* Virtual Interrupt Flag */ #define X86_EFLAGS_VIP 0x00100000 /* Virtual Interrupt Pending */ #define X86_EFLAGS_ID 0x00200000 /* CPUID detection flag */ /* * Generic CPUID function */ static inline void cpuid(int op, int *eax, int *ebx, int *ecx, int *edx) { __asm__("cpuid" : "=a" (*eax), "=b" (*ebx), "=c" (*ecx), "=d" (*edx) : "0" (op)); } /* * CPUID functions returning a single datum */ static inline unsigned int cpuid_eax(unsigned int op) { unsigned int eax; __asm__("cpuid" : "=a" (eax) : "0" (op) : "bx", "cx", "dx"); return eax; } static inline unsigned int cpuid_ebx(unsigned int op) { unsigned int eax, ebx; __asm__("cpuid" : "=a" (eax), "=b" (ebx) : "0" (op) : "cx", "dx" ); return ebx; } static inline unsigned int cpuid_ecx(unsigned int op) { unsigned int eax, ecx; __asm__("cpuid" : "=a" (eax), "=c" (ecx) : "0" (op) : "bx", "dx" ); return ecx; } static inline unsigned int cpuid_edx(unsigned int op) { unsigned int eax, edx; __asm__("cpuid" : "=a" (eax), "=d" (edx) : "0" (op) : "bx", "cx"); return edx; } #define load_cr3(pgdir) \ asm volatile("movl %0,%%cr3": :"r" (true_phys(__pa(pgdir)))) /* * Intel CPU features in CR4 */ #define X86_CR4_VME 0x0001 /* enable vm86 extensions */ #define X86_CR4_PVI 0x0002 /* virtual interrupts flag enable */ #define X86_CR4_TSD 0x0004 /* disable time stamp at ipl 3 */ #define X86_CR4_DE 0x0008 /* enable debugging extensions */ #define X86_CR4_PSE 0x0010 /* enable page size extensions */ #define X86_CR4_PAE 0x0020 /* enable physical address extensions */ #define X86_CR4_MCE 0x0040 /* Machine check enable */ #define X86_CR4_PGE 0x0080 /* enable global pages */ #define X86_CR4_PCE 0x0100 /* enable performance counters at ipl 3 */ #define X86_CR4_OSFXSR 0x0200 /* enable fast FPU save and restore */ #define X86_CR4_OSXMMEXCPT 0x0400 /* enable unmasked SSE exceptions */ /* * Save the cr4 feature set we're using (ie * Pentium 4MB enable and PPro Global page * enable), so that any CPU's that boot up * after us can get the correct flags. */ extern unsigned long mmu_cr4_features; static inline void set_in_cr4 (unsigned long mask) { mmu_cr4_features |= mask; __asm__("movl %%cr4,%%eax\n\t" "orl %0,%%eax\n\t" "movl %%eax,%%cr4\n" : : "irg" (mask) :"ax"); } static inline void clear_in_cr4 (unsigned long mask) { mmu_cr4_features &= ~mask; __asm__("movl %%cr4,%%eax\n\t" "andl %0,%%eax\n\t" "movl %%eax,%%cr4\n" : : "irg" (~mask) :"ax"); } /* * NSC/Cyrix CPU configuration register indexes */ #define CX86_PCR0 0x20 #define CX86_GCR 0xb8 #define CX86_CCR0 0xc0 #define CX86_CCR1 0xc1 #define CX86_CCR2 0xc2 #define CX86_CCR3 0xc3 #define CX86_CCR4 0xe8 #define CX86_CCR5 0xe9 #define CX86_CCR6 0xea #define CX86_CCR7 0xeb #define CX86_PCR1 0xf0 #define CX86_DIR0 0xfe #define CX86_DIR1 0xff #define CX86_ARR_BASE 0xc4 #define CX86_RCR_BASE 0xdc /* * NSC/Cyrix CPU indexed register access macros */ #define getCx86(reg) ({ outb((reg), 0x22); inb(0x23); }) #define setCx86(reg, data) do { \ outb((reg), 0x22); \ outb((data), 0x23); \ } while (0) /* * Bus types (default is ISA, but people can check others with these..) */ extern int MCA_bus; static inline void __monitor(const void *eax, unsigned long ecx, unsigned long edx) { /* "monitor %eax,%ecx,%edx;" */ asm volatile( ".byte 0x0f,0x01,0xc8;" : :"a" (eax), "c" (ecx), "d"(edx)); } static inline void __mwait(unsigned long eax, unsigned long ecx) { /* "mwait %eax,%ecx;" */ asm volatile( ".byte 0x0f,0x01,0xc9;" : :"a" (eax), "c" (ecx)); } /* from system description table in BIOS. Mostly for MCA use, but others may find it useful. */ extern unsigned int machine_id; extern unsigned int machine_submodel_id; extern unsigned int BIOS_revision; extern unsigned int mca_pentium_flag; /* * User space process size: 3GB (default). */ #define TASK_SIZE (PAGE_OFFSET) /* This decides where the kernel will search for a free chunk of vm * space during mmap's. */ #define TASK_UNMAPPED_BASE (PAGE_ALIGN(TASK_SIZE / 3)) #define HAVE_ARCH_PICK_MMAP_LAYOUT /* * Size of io_bitmap. */ #define IO_BITMAP_BITS 65536 #define IO_BITMAP_BYTES (IO_BITMAP_BITS/8) #define IO_BITMAP_LONGS (IO_BITMAP_BYTES/sizeof(long)) #define IO_BITMAP_OFFSET offsetof(struct tss_struct,io_bitmap) #define INVALID_IO_BITMAP_OFFSET 0x8000 #define INVALID_IO_BITMAP_OFFSET_LAZY 0x9000 struct i387_fsave_struct { long cwd; long swd; long twd; long fip; long fcs; long foo; long fos; long st_space[20]; /* 8*10 bytes for each FP-reg = 80 bytes */ long status; /* software status information */ }; struct i387_fxsave_struct { unsigned short cwd; unsigned short swd; unsigned short twd; unsigned short fop; long fip; long fcs; long foo; long fos; long mxcsr; long mxcsr_mask; long st_space[32]; /* 8*16 bytes for each FP-reg = 128 bytes */ long xmm_space[32]; /* 8*16 bytes for each XMM-reg = 128 bytes */ long padding[56]; } __attribute__ ((aligned (16))); struct i387_soft_struct { long cwd; long swd; long twd; long fip; long fcs; long foo; long fos; long st_space[20]; /* 8*10 bytes for each FP-reg = 80 bytes */ unsigned char ftop, changed, lookahead, no_update, rm, alimit; struct info *info; unsigned long entry_eip; }; union i387_union { struct i387_fsave_struct fsave; struct i387_fxsave_struct fxsave; struct i387_soft_struct soft; }; typedef struct { unsigned long seg; } mm_segment_t; struct thread_struct; struct tss_struct { unsigned short back_link,__blh; unsigned long esp0; unsigned short ss0,__ss0h; unsigned long esp1; unsigned short ss1,__ss1h; /* ss1 is used to cache MSR_IA32_SYSENTER_CS */ unsigned long esp2; unsigned short ss2,__ss2h; unsigned long __cr3; unsigned long eip; unsigned long eflags; unsigned long eax,ecx,edx,ebx; unsigned long esp; unsigned long ebp; unsigned long esi; unsigned long edi; unsigned short es, __esh; unsigned short cs, __csh; unsigned short ss, __ssh; unsigned short ds, __dsh; unsigned short fs, __fsh; unsigned short gs, __gsh; unsigned short ldt, __ldth; unsigned short trace, io_bitmap_base; /* * The extra 1 is there because the CPU will access an * additional byte beyond the end of the IO permission * bitmap. The extra byte must be all 1 bits, and must * be within the limit. */ unsigned long io_bitmap[IO_BITMAP_LONGS + 1]; /* * Cache the current maximum and the last task that used the bitmap: */ unsigned long io_bitmap_max; struct thread_struct *io_bitmap_owner; /* * pads the TSS to be cacheline-aligned (size is 0x100) */ unsigned long __cacheline_filler[35]; /* * .. and then another 0x100 bytes for emergency kernel stack */ unsigned long stack[64]; } __attribute__((packed)); #define ARCH_MIN_TASKALIGN 16 struct thread_struct { /* cached TLS descriptors. */ struct desc_struct tls_array[GDT_ENTRY_TLS_ENTRIES]; unsigned long esp0; unsigned long sysenter_cs; unsigned long eip; unsigned long esp; unsigned long fs; unsigned long gs; /* Hardware debugging registers */ unsigned long debugreg[8]; /* %%db0-7 debug registers */ /* fault info */ unsigned long cr2, trap_no, error_code; /* floating point info */ union i387_union i387; /* virtual 86 mode info */ struct vm86_struct __user * vm86_info; unsigned long screen_bitmap; unsigned long v86flags, v86mask, saved_esp0; unsigned int saved_fs, saved_gs; /* IO permissions */ unsigned long *io_bitmap_ptr; /* max allowed port in the bitmap, in bytes: */ unsigned long io_bitmap_max; }; #define INIT_THREAD { \ .vm86_info = NULL, \ .sysenter_cs = __KERNEL_CS, \ .io_bitmap_ptr = NULL, \ } /* * Note that the .io_bitmap member must be extra-big. This is because * the CPU will access an additional byte beyond the end of the IO * permission bitmap. The extra byte must be all 1 bits, and must * be within the limit. */ #define INIT_TSS { \ .esp0 = sizeof(init_stack) + (long)&init_stack, \ .ss0 = __KERNEL_DS, \ .ss1 = __KERNEL_CS, \ .ldt = GDT_ENTRY_LDT, \ .io_bitmap_base = INVALID_IO_BITMAP_OFFSET, \ .io_bitmap = { [ 0 ... IO_BITMAP_LONGS] = ~0 }, \ } static inline void load_esp0(struct tss_struct *tss, struct thread_struct *thread) { tss->esp0 = thread->esp0; /* This can only happen when SEP is enabled, no need to test "SEP"arately */ if (unlikely(tss->ss1 != thread->sysenter_cs)) { tss->ss1 = thread->sysenter_cs; wrmsr(MSR_IA32_SYSENTER_CS, thread->sysenter_cs, 0); } } #define start_thread(regs, new_eip, new_esp) do { \ __asm__("movl %0,%%fs ; movl %0,%%gs": :"r" (0)); \ set_fs(USER_DS); \ regs->xds = __USER_DS; \ regs->xes = __USER_DS; \ regs->xss = __USER_DS; \ regs->xcs = __USER_CS; \ regs->eip = new_eip; \ regs->esp = new_esp; \ } while (0) /* Forward declaration, a strange C thing */ struct task_struct; struct mm_struct; /* Free all resources held by a thread. */ extern void release_thread(struct task_struct *); /* Prepare to copy thread state - unlazy all lazy status */ extern void prepare_to_copy(struct task_struct *tsk); /* * create a kernel thread without removing it from tasklists */ extern int kernel_thread(int (*fn)(void *), void * arg, unsigned long flags); extern unsigned long thread_saved_pc(struct task_struct *tsk); void show_trace(struct task_struct *task, unsigned long *stack); unsigned long get_wchan(struct task_struct *p); #define THREAD_SIZE_LONGS (THREAD_SIZE/sizeof(unsigned long)) #define KSTK_TOP(info) \ ({ \ unsigned long *__ptr = (unsigned long *)(info); \ (unsigned long)(&__ptr[THREAD_SIZE_LONGS]); \ }) #define task_pt_regs(task) \ ({ \ struct pt_regs *__regs__; \ __regs__ = (struct pt_regs *)KSTK_TOP((task)->thread_info); \ __regs__ - 1; \ }) #define KSTK_EIP(task) (task_pt_regs(task)->eip) #define KSTK_ESP(task) (task_pt_regs(task)->esp) struct microcode_header { unsigned int hdrver; unsigned int rev; unsigned int date; unsigned int sig; unsigned int cksum; unsigned int ldrver; unsigned int pf; unsigned int datasize; unsigned int totalsize; unsigned int reserved[3]; }; struct microcode { struct microcode_header hdr; unsigned int bits[0]; }; typedef struct microcode microcode_t; typedef struct microcode_header microcode_header_t; /* microcode format is extended from prescott processors */ struct extended_signature { unsigned int sig; unsigned int pf; unsigned int cksum; }; struct extended_sigtable { unsigned int count; unsigned int cksum; unsigned int reserved[3]; struct extended_signature sigs[0]; }; /* '6' because it used to be for P6 only (but now covers Pentium 4 as well) */ #define MICROCODE_IOCFREE _IO('6',0) /* REP NOP (PAUSE) is a good thing to insert into busy-wait loops. */ static inline void rep_nop(void) { __asm__ __volatile__("rep;nop": : :"memory"); } #define cpu_relax() rep_nop() /* generic versions from gas */ #define GENERIC_NOP1 ".byte 0x90\n" #define GENERIC_NOP2 ".byte 0x89,0xf6\n" #define GENERIC_NOP3 ".byte 0x8d,0x76,0x00\n" #define GENERIC_NOP4 ".byte 0x8d,0x74,0x26,0x00\n" #define GENERIC_NOP5 GENERIC_NOP1 GENERIC_NOP4 #define GENERIC_NOP6 ".byte 0x8d,0xb6,0x00,0x00,0x00,0x00\n" #define GENERIC_NOP7 ".byte 0x8d,0xb4,0x26,0x00,0x00,0x00,0x00\n" #define GENERIC_NOP8 GENERIC_NOP1 GENERIC_NOP7 /* Opteron nops */ #define K8_NOP1 GENERIC_NOP1 #define K8_NOP2 ".byte 0x66,0x90\n" #define K8_NOP3 ".byte 0x66,0x66,0x90\n" #define K8_NOP4 ".byte 0x66,0x66,0x66,0x90\n" #define K8_NOP5 K8_NOP3 K8_NOP2 #define K8_NOP6 K8_NOP3 K8_NOP3 #define K8_NOP7 K8_NOP4 K8_NOP3 #define K8_NOP8 K8_NOP4 K8_NOP4 /* K7 nops */ /* uses eax dependencies (arbitary choice) */ #define K7_NOP1 GENERIC_NOP1 #define K7_NOP2 ".byte 0x8b,0xc0\n" #define K7_NOP3 ".byte 0x8d,0x04,0x20\n" #define K7_NOP4 ".byte 0x8d,0x44,0x20,0x00\n" #define K7_NOP5 K7_NOP4 ASM_NOP1 #define K7_NOP6 ".byte 0x8d,0x80,0,0,0,0\n" #define K7_NOP7 ".byte 0x8D,0x04,0x05,0,0,0,0\n" #define K7_NOP8 K7_NOP7 ASM_NOP1 #ifdef CONFIG_MK8 #define ASM_NOP1 K8_NOP1 #define ASM_NOP2 K8_NOP2 #define ASM_NOP3 K8_NOP3 #define ASM_NOP4 K8_NOP4 #define ASM_NOP5 K8_NOP5 #define ASM_NOP6 K8_NOP6 #define ASM_NOP7 K8_NOP7 #define ASM_NOP8 K8_NOP8 #elif defined(CONFIG_MK7) #define ASM_NOP1 K7_NOP1 #define ASM_NOP2 K7_NOP2 #define ASM_NOP3 K7_NOP3 #define ASM_NOP4 K7_NOP4 #define ASM_NOP5 K7_NOP5 #define ASM_NOP6 K7_NOP6 #define ASM_NOP7 K7_NOP7 #define ASM_NOP8 K7_NOP8 #else #define ASM_NOP1 GENERIC_NOP1 #define ASM_NOP2 GENERIC_NOP2 #define ASM_NOP3 GENERIC_NOP3 #define ASM_NOP4 GENERIC_NOP4 #define ASM_NOP5 GENERIC_NOP5 #define ASM_NOP6 GENERIC_NOP6 #define ASM_NOP7 GENERIC_NOP7 #define ASM_NOP8 GENERIC_NOP8 #endif #define ASM_NOP_MAX 8 /* Prefetch instructions for Pentium III and AMD Athlon */ /* It's not worth to care about 3dnow! prefetches for the K6 because they are microcoded there and very slow. However we don't do prefetches for pre XP Athlons currently That should be fixed. */ #define ARCH_HAS_PREFETCH extern inline void prefetch(const void *x) { alternative_input(ASM_NOP4, "prefetchnta (%1)", X86_FEATURE_XMM, "r" (x)); } #define ARCH_HAS_PREFETCH #define ARCH_HAS_PREFETCHW #define ARCH_HAS_SPINLOCK_PREFETCH /* 3dnow! prefetch to get an exclusive cache line. Useful for spinlocks to avoid one state transition in the cache coherency protocol. */ extern inline void prefetchw(const void *x) { alternative_input(ASM_NOP4, "prefetchw (%1)", X86_FEATURE_3DNOW, "r" (x)); } #define spin_lock_prefetch(x) prefetchw(x) extern void select_idle_routine(const struct cpuinfo_x86 *c); #define cache_line_size() (boot_cpu_data.x86_cache_alignment) #endif /* __ASM_I386_PROCESSOR_H */ |
|
From: Itsuro O. <od...@us...> - 2005-12-15 04:27:29
|
Update of /cvsroot/mkdump/minikpatch/1.0/linus/2.6.9/init In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv8413/1.0/linus/2.6.9/init Log Message: Directory /cvsroot/mkdump/minikpatch/1.0/linus/2.6.9/init added to the repository |
|
From: Itsuro O. <od...@us...> - 2005-12-15 04:27:13
|
Update of /cvsroot/mkdump/minikpatch/1.0/linus/2.6.9/include/linux In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv8390/1.0/linus/2.6.9/include/linux Log Message: Directory /cvsroot/mkdump/minikpatch/1.0/linus/2.6.9/include/linux added to the repository |
|
From: Itsuro O. <od...@us...> - 2005-12-15 04:25:43
|
Update of /cvsroot/mkdump/minikpatch/1.0/linus/2.6.9/include/asm-x86_64 In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv8257/1.0/linus/2.6.9/include/asm-x86_64 Log Message: Directory /cvsroot/mkdump/minikpatch/1.0/linus/2.6.9/include/asm-x86_64 added to the repository |
|
From: Itsuro O. <od...@us...> - 2005-12-15 04:25:24
|
Update of /cvsroot/mkdump/minikpatch/1.0/linus/2.6.9/include/asm-i386 In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv8236/1.0/linus/2.6.9/include/asm-i386 Log Message: Directory /cvsroot/mkdump/minikpatch/1.0/linus/2.6.9/include/asm-i386 added to the repository |
|
From: Itsuro O. <od...@us...> - 2005-12-15 04:25:09
|
Update of /cvsroot/mkdump/minikpatch/1.0/linus/2.6.9/include In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv8189/1.0/linus/2.6.9/include Log Message: Directory /cvsroot/mkdump/minikpatch/1.0/linus/2.6.9/include added to the repository |
|
From: Itsuro O. <od...@us...> - 2005-12-15 04:20:48
|
Update of /cvsroot/mkdump/minikpatch/1.0/linus/2.6.9/fs In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv7817/1.0/linus/2.6.9/fs Log Message: Directory /cvsroot/mkdump/minikpatch/1.0/linus/2.6.9/fs added to the repository |
|
From: Itsuro O. <od...@us...> - 2005-12-15 04:20:26
|
Update of /cvsroot/mkdump/minikpatch/1.0/linus/2.6.9/drivers/block In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv7768/1.0/linus/2.6.9/drivers/block Log Message: Directory /cvsroot/mkdump/minikpatch/1.0/linus/2.6.9/drivers/block added to the repository |
|
From: Itsuro O. <od...@us...> - 2005-12-15 04:20:17
|
Update of /cvsroot/mkdump/minikpatch/1.0/linus/2.6.9/drivers In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv7739/1.0/linus/2.6.9/drivers Log Message: Directory /cvsroot/mkdump/minikpatch/1.0/linus/2.6.9/drivers added to the repository |
|
From: Itsuro O. <od...@us...> - 2005-12-15 04:20:03
|
Update of /cvsroot/mkdump/minikpatch/1.0/linus/2.6.9/arch/x86_64/mm In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv7629/1.0/linus/2.6.9/arch/x86_64/mm Log Message: Directory /cvsroot/mkdump/minikpatch/1.0/linus/2.6.9/arch/x86_64/mm added to the repository |
|
From: Itsuro O. <od...@us...> - 2005-12-15 04:19:50
|
Update of /cvsroot/mkdump/minikpatch/1.0/linus/2.6.9/arch/x86_64/kernel In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv7603/1.0/linus/2.6.9/arch/x86_64/kernel Log Message: Directory /cvsroot/mkdump/minikpatch/1.0/linus/2.6.9/arch/x86_64/kernel added to the repository |
6 messages has been excluded from this view by a project administrator.
